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March 2, 1994 

The Honorable Nancy Landon Kassebaum 
Ranking Minority Member 
Committee on Labor and Human Resources 
United States Senate 

Dear Senator Kassebaum: 

A strong internationally competitive economy depends, in part, on 
effectively preparing workers to compete in the workforce. Towards this 
end, the federal government has invested considerable effort and 
resources in (1) facilitating potential workers' entry into the workforce, 
(2) helping workers overcome barriers that hamper their ability to 
compete for jobs, and (3) assisting dislocated workers in reentering the 
workforce. We have recently identified^ over 150 programs tiiat provide 
employment training assistance.^ To know whether these programs are 
getting the most for the resources invested, federal agencies not only need 
to track the expenditure of resources, but also to determine what the 
outcomes were for participants and whether government programs made a 
difference for those who received services. 

To learn more about how federal agencies assjsss whether their 
employment training programs are working, you asked us to determine 

• what data federal agencies collect on participant outcomes, 

• how federal agencies monitor local program performance, and 

• what studies of program effectiveness have been conducted. 

To accomplish these objectives, we focused on programs that pro\dde 
employment training assistance to the economically disadvantaged. 
According to the proposed fiscal year 1994 budget, 9 programs specifically 
target the economically disadvantaged, but a total oi 62 programs,^ 



' Multiple Employment Programs; National Employment Training Strategy Needed (GAOn'-HRD-93-27, 
June 18, 1993). 

^As used in this report, "employment training programs'* refers to programs identified as (1) providing 
assistance to the unemployed, (2) creating employment opportunities, and (3) enhancing the skills of 
participants to increase their employability. We only included those programs that provide services to 
adults and out-of-school youth not enrolled in advanced-degree programs. 

^ur analysis was originally based on 65 programs targeting the economically disadvantaged existing 
in fiscal year 1991. Since that time, some programs have been dropped while others were added or 
consolidated into other programs. The total number of programs included in our analysis is 62. A 
complete listing of the 154 programs or funding streams identified as providing employment training 
assistance is shown in appendix I. 
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admirdstered by 14 different federal agencies, provide some employment 
training assistance to the economically disadvantaged. These 62 programs 
had proposed budgets of about $17 billion^ or about two-thirds of the 
federal funds spent on employment training assistance. During our review, 
we 

interviewed agency officials and obtained copies of data collected at the 
federal level, 

reviewed agency monitoring reports to determine the extent of oversight 
provided and whether agencies monitor program outcomes, and 
asked agency officials to identify studies conducted during the 10-year 
period, ending December 1993, concerning the effectiveness of any of the 
62 programs in our analysis, and did a literature search, using 
governmental and commercially produced databases, to check fc 
additional studies published during the period January 1990 througli 
December 1993.^ 

Because the focus of our analysis was on identifying which agencies 
collect data on participant outcomes or measure program effectiveness, 
we did not attempt to determine why some agencies did not collect this 
information or conduct studies that measure program effectiveness. 

In addition to this report, we have issued two other reports concerning 
problems with the current fragmented "system** of multiple employment 
training programs. The first report, Multiple Employment Training 
Programs: Conflicting Requirements Hamper Delivery of Services 
(GAO/HEHS-94-78, Jan. 28, 1994), concerns the extent to which differences in 
eligibility criteria a:ad annual operating cycles hamper the ability of 
employment training programs to provide needed services. The second 
report, Multiple Employment Training Programs: Overlapping Programs 
Can Add Unnecessary Costs (gao/hehs-94^0, Jan. 28, 1994), concerns the 
extent to which programs overlap in the populations they target, in the 
services they provide, and in the administrative structures they create to 
deliver those services. We also have other ongoing work that will address 
the need for a major overhaul of the entire federal employment training 
system.. 



^Budget estimates are primarily based on the President's proposed budget for fiscal year 1994, dated 
April 8, 1993. When information was available » numbers in this total have been adUusted to represent 
only that portion of the program that provided assistance to adults or out-of-school youth not enrolled 
in advanced-degree programs. 

^Using this approach, we were able to identify over 90 studies that wer^; characterized as management 
or effectiveness studies; however, we may not have identified all the studies that had been published 
during that period. 
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Background 



The federal government has historically played an important role in 
providing employment training assistance to the economically 
disadvantaged. In fiscal year 1994, federal programs are expected to invest 
an estimated $17 billion on employment training assistance for the 
economically disadvantaged, including (1) counseling and assessment, 
(2) remedial education, (3) vocational skill training, (4) placement 
assistance, and (5) support services. This assistance is provided through 
14 federal departments and independent agencies, including agencies in 
the Departments of Education, Health and Human Services, Housing and 
Urban Development, and Labor, as well as independent agencies, such as 
Action, the Small Business Administration, and the Office of Personnel 
Management. 



To ensure programs get the most from this investment, federal program 
administrators must have information about their programs* strengths and 
weaknesses. With this information, they can suggest changes to improve 
programs, such as modifying the types or number of services available, to 
help participants receive training that meets their needs and enables them 
to obtain employment. 

The Congress recently passed the Government Performance and Results 
Act of 1993, which will require agencies to gather program performance 
data. Specifically, the act requires agencies to (1) have a strategic plan for 
program activities; (2) establish program performance goals that are 
objective, quantifiable, and measurable; and (3) submit a report on 
program performance to the President and the Congress. 



Results in Brief 



Federal agencies closely monitor their expenditure of billions of dollars 
for employment training assistance for the economically disadvantaged. 
However, most agencies do not collect information on participant 
outcomes nor do they conduct studies of program effectiveness — both of 
which are needed to know how well programs are helping participants 
enter or reenter the workforce. As a result, these agencies do not know 
whether their programs, as currently configured, are providing assistance 
that results in participants getting jobs. Even when participants got jobs, 
agencies do not know whether employment resulted from participation in 
the program or if participants would most likely have found the same 
types of jobs on their ovm^ without federal assistance. 



For about half the programs in our analysis, agencies did not collect data 
on what happened to program participants after they completed a given 
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program — neither whether they obtained jobs nor what wages they 
earned. The size of the program did not appear to make a difference in 
whether participant outcome data were collected. Large programs with 
annual budgets over $100 million were no more likely to have collected 
data on participant outcomes than smaller programs with budgets under 
$50 million. Without this information, admixiistrators of programs large 
and small have difficulty knowing whether they are training participants 
for real job opportunities and whether participants have the skills 
employers need. 

In addition, a] though most of the employment training programs in our 
analysis had some form of oversight or monitoring effort, only about a 
third cS the programs \ised these activities to assess participant outcomes. 
Again, the size of the program appeared to make little difference in the 
focus of these activities. Most oversight efforts focused only on 
compliance with program requirements and procedures or the amount of 
progress being made to provide agreed-upon services. Therefore, although 
administrators may be assured that their programs were in compliance, 
they did not know whether these programs achieved the results intended. 

The federal agencies responsible for these programs seldom conducted 
studies that measure program effectiveness or impact. According to 
program administrators, during the last 10 years, only 7 of the 62 programs 
in our analysis had studies that evaluated whether these programs made a 
difference for the participants, that is, whether participants would most 
likely have achieved the same outcomes without assistance. The studies 
that were performed tended to address larger programs, with 5 of the 7 
programs having annual budgets over $100 million. But even among the 
larger programs, these studies addressed programs that accounted for only 
16 percent of the proposed fiscal year 1994 funding for the 15 employment 
training assistance programs with budgets over $100 million. 



Almost all of the programs we reviewed collected some data on the dollars 
spent, services provided, and number of participants served. But only 
about half the programs collected outcome data on what happened to 
program participants after they received program services. As a result, 
administrators did not know if participants got jobs; if they got jobs, 
administrators did not know whether the jobs were related to the training 
provided or what wages the participants earned. Without this information, 
program administrators have difficulty determining if they are preparing 
paiticipants for looal labor market opportunities and whether participants 



Many Programs Do 
Not Collect Data on 
Whether Participants 
Obtained Jobs 
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gain the skills needed to meet employer requirements. For example, 
becaiise the Food Stamp Employment and Training program, as is the case 
with many other programs, does not collect data on who obtains a job 
after participating in the program, program administrators camiot assess 
whether the program appears to be achieving its objectives of helping the 
economically disadvantaged find employment or become self-sufficient. 

As shown in figure 1, of the 62 programs that provide assistance to the 
economically disadvantaged, about 90 percent of the programs collected 
data on dollars spent and the number of participants served. However, 
only 49 percent of the programs collected data on how many participants 
obtained jobs and only 26 percent collected data on wages earned. Size of 
the program did not appear to be a factor in determining which programs 
collected participant outcome data. Smaller programs vdth annual budgets 
of less than $50 million were just as likely as large programs with budgets 
of $100 million or more to collect participant outcome data. The categories 
of outcome data collected for each program in our analysis are shown in 
appendix U. 
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Figure 1: Percent of Programs Collecting Various Core Data Elements 
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Even when outcome data were collected, few linked these data with data 
on the services provided or participant demographic characteristics. We 
found that only about 34 percent of the programs serving the economically 
disadvantaged attempted to link the outcomes achieved by participants 
with services provided or demographic characteristics. For example, the 
Even Start-Migrant Education program collected data on participant 
employment status and wages earned, which can be used to determine if 
the program is meeting its goals. However, the program did not link 
participant outcome data with data on services or training provided. As a 
result, administrators of the program did not know whether the training 
they provided helped participants obtain jobs, nor could they identify ways 
to modify the program to improve performance. 

Program officials shoudd know whether participants trained as truck 
drivers, for example, get jobs as truck drivers. By linking participant 
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demographic characteristics to training provided and job outcomes, 
program officials can determine whether their programs are more 
successful for some participants (for example, men) than others (for 
example, women). Officials also can determine whether there are 
disparities in who receives what types of training. For example, in our 
report on racial and gender disparities in Job Training Partnership Act 
(jtpa) services, we reported that 34 percent of the projects in our analysis 
(67 of 199) had a disparity in at least one training mode — classroom 
training, on-the-job training, or job search assistance only — ^for at least one 
of the racial groups assessed.^ 



Monitoring Activities 
Generally Ignore 
Program Outcomes 



Most employment training programs we reviewed had some form of 
monitoring or oversight. However, these monitoring efforts generally 
concerned compliance with program requirements and procedures, such 
as compliance with Equal Employment Opportunity (eeo) regulations, or 
progress made in providing agreed-upon services, such as providing 
classrooms for specific training activities. These efforts did not include 
participant outcomes. 



ERIC 



Altiiough compliance with program requirements and financial integrity 
are important concerns, the failure to consider participant outcomes as a 
part of agencies' strategies for planning oversight efforts results in 
agencies' not being able to identify local projects that are having 
performance successes or difficulties. For example, one monitoring report 
we reviewed verified that the participants listed by the local project were 
actually served and were satisfied with the services received. The report, 
however, did not include what happen .»d to the participants after they 
received these services or whether the services provided helped them find 
jobs. As a result, although program administrators were assured resources 
had been used correctly, they did not know whether those resources 
achieved the results intended. 

As shown in table 1, 97 percent of the 62 programs providing employment 
training assistance to the economically disadvantaged had some form of 
federal monitoring or oversight, but only 34 percent of the oversight 
efforts we identified included an assessment of participant outcomes. 
Program size did not appear to be a factor in whether monitoring included 
an assessment of participant outcomes. Larger programs were no more 



^ Job Training Partnership Act: Racial and Gender Disparities in Services (GAO/HRD-9M48, Sept 20, 
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likely to have used their oversight activities to monitor participant 
outcomes than smaller programs. 



Table 1: Percentage of Programs by 
Type of Monitoring Activities 





Percentage of 


Type of monitoring 


programs 


Compliance/status^ 


97 


Financial 


73 


Participant outcomes 


34 



progress made in providing agreed-upon services. 



Even when federal staff visit local projects, they concentrate their 
assessment on compliance issues rather than participant outcomes. We 
found that of the 62 prograntis serving the economically disadvantaged, 
87 percent had site visits usually performed by federal staff in field offices. 
However, only 37 percent looked at participant outcomes when they 
visited local projects. The areas covered by the monitoring activities of 
each of the 62 programs in our analysis are shown in appendix in. 



Few Studies Have 
Been Performed 
Concerning Program 
Effectiveness or 
Impact 



Tracking participant outcomes can provide important feedback to 
agencies on the extent to which programs are achieving their objectives. 
But to determine whether programs providing employment training 
assistance are really making a difference^ or whether participants would 
most likely have achieved the same outcomes without the program, 
agencies must compare the outcomes achieved by program participants 
with the outcomes of similar nonparticipants. However, we found that few 
agencies operating the 62 programs that served the economically 
disadvantaged had sponsored such studies. Program administrators 
identified only 7 programs that had been studied, during the 10-year period 
ending December 1993, using a comparison of participant outcomes with 



^The terms program effectiveness and program impact are interchangeable as used in our analysis. We 
define "program effectiveness evaluation** as the application of scientific research methods to estimate 
how much of observed results, intended or not, are caused by program activities. Effect is linked to 
cause by design and analysis that compare observed results with estimates of what might have been 
observed in the absence of the program. In the textbook Evaluation; A Systematic Approach . Peter H. 
Rossi and Howard E. Freaman define the term impact assessment as "Evaluation of the extent to 
which a program causes changes in the desired direction in the target population... .Impact assessment 
is directed at establishing, with as much certainty as possible, whether or not an intervention is 
producing its intended effects....'nie outcomes of social programs are assessed by comparing 
information about participants and nonparticipants, before and after an intervention, or by other less 
powerful research designs. But the essential considerations involve the systematic rejection of 
alternative, competing explanations for the observed outcomes other than the intervention." 
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the outxjomes of similar nonparticipants. See table 2 for a listing of 
programs and related studies. 

Our analysis of the programs that had been studied showed that larger 
programs were more likely to have been the subject of these studies. We 
found that 5 of the 7 programs studied had annual budgets over 
$100 million. But even among the larger programs, the programs studied 
only accoimted for 16 percent of the total proposed funding for the 15 
programs with budgets over $100 million. 

While we did not determine why more agencies did not perform 
effectiveness studies using a comparative approach, the Department of 
Labor, in commenting on the report, stated that the high costs and the 
denial of potentially beneficial services were two reasons why more 
agencies have not used this approach to assess the effectiveness of their 
programs. While other approaches may be more feasible, without random 
assignment, conclusive attribution of effects to various treatments caimot 
be made. 
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Table 2: Studies Published Between ^^^^^^■■^^^^^^{^■^^■^^■^^^^■^^^^^^^^^■iH 

January 1990 and December 1993 That Program/Study Title/Author of Study/Date 

Evaluated Program Effectiveness Food Stamp Employment and Training Program (Agriculture) 

Evaluation of the Food Stamp Employment Program, Abt. Associates, Inc., June 1990 

Even Start (Education) 

National Evaluation of the Even Start Family Literacy Program: Report on Effectiveness, 
Abt. Associates. Inc., and RMC Research Corporation, October 1993 

Job Opportunities and Basic Skills Program (Health and Human Services) 

GAIM: Two-year Impacts in Six Counties— California's Greater Avenues for 
Independence Program, Manpower Demonstration Research Corporation, 
May 1993 

Community Service Block Grant, Demonstrations (Heaith and Human Services) 

Summary of Final Evaluation Findings from FY89: Demonstration Partnership Program 
Projects. Monograph Series 100-89: Case Management Family Intervention Models, 
Department of Health and Human Services, June 1992 

Summary of Final Evaluation Findings from FY89: Demonstration. Partnership 
Program Projects. Monograph Series 200-89: Micro-Business and Self-Employment, 
Department of Health and Human Services. June 1992 

Summary of Final Evaluation Findings from FY89: Demonstration Partnership 
Program Projects. Monograph Series 300-89: Homeless Individuals and Families, 
Department of Health and Human Services, June 1992 

Summary of Final Evaluation Findings from FY89: Demonstration Partnership 
Program Projects. Monograph Series 400-89: Early Prevention-High School 
Youth-at-Risk. Department of Health and Human Services. June 1992 

JTPA ilA - Disadvantaged Adults (Labor) 

The National JTPA Study: Title ll-A Impacts on Earnings and Employment at 18 Months, 
Abt. Associates, Inc., January 1993 

Evaluating JTPA Programs for Economically Disadvantaged Adults: A Case Study of 
Utah and General Findings Research Report, National Commission for Employment 
Policy, June 1993 

Employment Services (Labor) 

Labor Market Implications of ES Services for Duration of Joblessness, Probability of 
Subsequently Remaining Employed, and Repeated Spells of Joblessness: 
Comparison of Ul Beneficiaries, W.E. Upjohn Institute for Employment Research, 
April 1993 

Trade Adjustment Assistance Worlcers (Labor) 

International Trade anH Worker Dislocation: Evaluation of the Trade Adjustment 
Assistance Program, Mathematica Policy Research, Inc., April 1993 



Our review of studies that were published during the 4-year period, 
January 1990 through December 1993, identified many additional studies 
that were characterized as management or effectiveness studies. While 
these studies may provide program administrators useful information on 
their programs, they do not provide a statistically valid approach for 
evaluating program effectiveness. We found, however, that most of the 
studies focused on resource or program management issues, such as 
program costs, number and characteristics of participants served, and 
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types of services provided. As shown in table 3, of 94 studies published 
during that 4-year period, 89 focused on various aspects of program 
management affecting 28 programs. Forty-five of the studies dealt with 
participant outcomes from 20 programs, of which 26 studies linked 
outcomes to ser/ices provided or participant characteristics. Eleven 
studies^ concerning 7 programs attempted to evaluate program 
effectiveness or impact by comparing participant outcomes to those of 
similar individuals not receiving program services. A complete list of the 
94 studies we identified and the subject areas covered are shown in 
appendix IV. 



Table 3: Studies by the Issues 
Evaluated 



Issue evaluated 


Studies 


Programs 


Resource management 


89 


28 


Participant outcomes 


45 


20 



Program effectiveness 



11 



Some studies indicate that the programs were successful in that outcomes 
achieved by program participants were better than the outcomes achieved 
by similar individuals wb'^ 1id not receive services from the programs. 
Other studies, however, *se questions about the value of the programs, 
as they are currently coufigured, because program participants did not 
achieve significantly better results than similar individuals who did not 
receive services. For example, the jtpa program outcome data have shown 
high placement rates and wage levels; however, a study of the jtpa 
program by Abt Associates, Inc., published in January 1993, has raised 
questions about the program's effectiveness. The study shows that while 
comparisons between program participants — adult women and men — had 
generally positive effects on earnings and employment compared with 
their counterparts in the control group, the jtpa program had little or no 
effect on female youth participants and male youth participants had lower 
earnings than their counterparts in the control group. The results of each 
study that assessed program effectiveness are summarized in appendix V. 
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^Education officials identified six studies now being conducted that would measure program 
effectiveness or impact; however, none of these studies will be available until later in 1994. The 
programs being evaluated include Upward Bound, Student Support Services, Adult Education, School 
Dropout Demonstration Assistance, National Workplace Literacy, and Direct Student Loan. 
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While many people are successful at finding jobs on their own, others have 
a more difficult time. To help the economically disadvantaged to compete 
in the workforce, the federal government expects to spend an estimated 
$17 billion in fiscal year 1994. Yet, most federal agencies do not collect the 
participant outcome data nor do they conduct the program effectiveness 
studies needed to provide them information on how well, if at all, their 
programs are actually helping people find jobs. 

Although we did not determine why agencies did not collect participant 
outcome data nor measure program effectiveness, it is clear that without 
this information, program administrators in these agencies cannot 
(1) determine how well programs appear to be doing in preparing people 
for employment, (2) identify what services are most effective in helping 
program participants obtain employment, (3) suggest the adjustments 
needed to improve services to get better results for the resources invested, 
or (4) determine what impact, if any, their programs have had in helping 
the economically disadvantaged to enter or reenter the workforce. 



Agency Comments 
and Our Response 



We sent a draft copy of this report to the following 14 departments and 
independent agencies for comment: Action, Department of Agriculture, 
Appalachian Regional Cormnission, Department of Commerce, 
Department of Defense, Department of Education, Department of Health 
and Human Services, Department of Housing and Urban Development, 
Department of the Interior, Department of Labor, the Office of Personnel 
Management, the Small Business Administration, Department of 
Transportation, and the Department of Veterans Affairs. Written 
comments were received from the Department of Defense, Department of 
Education, Department of Health and Human Services, Department of 
Labor, the Office of Personnel Management, and the Department of 
Transportation (see app. VI). Action, the Department of Agriculture, 
Department of Housing and Urban Development, Small Business 
Administration, and Department of Veterans Affairs provided us with oral 
comments. When appropriate, we used the cormnents received to clarify 
and update our report. The Appalachian Regional Commission, 
Department of Commerce, and Department of the Interior did not respond. 



While most of the comments received agreed with the overall conclusion 
of the report, they raised questions concerning the specific data in the 
report related to their programs. Their cormnents fell into three areas: 
(1) concerns about the inclusion of some programs in our analysis, (2) the 
use of proposed funding levels rather than actual levels, and (3) the 
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narrow criteria used txy identify studies of program effectiveness. We have 
summarized their comments and our response below. See appendix VI for 
the written comments received and our detailed response. 

Several agencies questioned the inclusion of some programs in the study 
because they believe their programs do not fit the scope of our analysis. 
Ouj analysis includes all programs and funding streams that provided 
assistance to adults and out-of-school youth to enhance their skills or 
employment opportunities. This is consistent with footnote 1 in appendix I 
that defines the federally funded programs included in our analysis as 
those that assist the unemployed, create employment, or enhance 
employability. We believe that the programs questioned by the Action 
agency. Department of Defense, Department of Education, Department of 
Health and Human Services, Department of Labor, and the Small Business 
Administration provide assistance that enhances participant employability 
or emplojnnent opportunities and, therefore, meet our criteria. 

For example, in their oral comments, the Action agency objected to the 
inclusion of the Literacy Corps, the Foster Grandparent, and Senior 
Companion Programs in our analysis. These programs provide volunteers 
to assist communities to resolve local poverty-related problems such as 
illiteracy, unemployment, and homelessness. Because economically 
disadvantaged volunteers in these programs receive an hourly stipend, 
these programs are considered to have provided employment 
opportunities for the economically disadvantaged. Both the National 
Commission for Employment Policy and the Congressional Research 
Service have included these programs in their studies of employment 
training assistance programs. The Literacy Corps also enhances the skills 
of economically disadvantaged people. As a result, we believe these 
programs should be included in our analysis. In addition to concerns fi:om 
the Action agency, the Small Business Administration questioned the 
inclusion of the Minority Business Development Program and the Business 
Development Assistance to Small Business Program in our analysis. To 
clarify what types of programs were included in the scope of our work, we 
have modified the description of our criteria. 

Several Departments and agencies also questioned the funding amounts 
cited in appendix L These amounts are based primarily on the proposed 
budget submitted by the President, April 8, 1993. We recognize that actual 
funding levels may reflect differences from the amounts shown in 
appendix I, but determining actual funding levels for so many different 
programs at one point in time was not practical. We have expanded our 



Page 13 



!5 

GAO/HEHS-94-88 Multiple Employment Tr&lning Programs 



B-266631 



explanation of the source of our funding data in the text of the report as 
well as in appendix 1. 

In some instances, the amounts cited in appendix I are only a portion of 
the proposed budget for that program because only a portion of the funds 
were used for adults or out-of-school youths. In those instances, when 
agency officials were able to provide us an estimate of how much should 
be included in the program, the amounts shown have been adjusted. In 
other instances, when the portion of funds that were used for adults and 
out-of-school youth could not be determined, we included proposed 
funding for the entire program. We recognize that our estimates of funding 
for employment training assistance are only an approximation. However, 
we believe the concern of the report is not funding levjels, but whether 
program administrators have the information needed to determine 
whether their programs are working effectively. 

Some agencies also suggested that additional studies should have been 
included in our analysis of agency efforts to detennine the effectiveness of 
their programs. We found that the additional studies mentioned (1) were 
still under way and we could not determine whether they will look at 
program effectiveness using analyses of participant outcomes compared 
with outcomes of similar groups of nonparticipants, (2) were already 
included in our analysis, or (3) did not appear to use the comparative 
analysis approach to look at program effectiveness. We recognize that our 
definition is rather strict. Our emphasis on the comparative approach 
should not be interpreted to suggest that studies that do not use a 
comparative approach do not provide useful information. Other 
techniques do indeed give indications of effectiveness. However, without 
random assignment, conclusive attribution of effects to various treatments 
cannot be made. 
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We did our work between February and November 1993 in accordance 
with generally accepted government auditing standards. As agreed, we 
obtained agency comments on this report. As requested, unless you 
publicly announce its contents earlier, we plan no further distribution of 
this report until 30 days from its issue date. At that time, we will send 
copies of this report to the agencies responsible for administering the 
programs we reviewed and to other interested parties. If you or your staff 
have any questions concerning this report, please call me at 
(202) 512-7014. Other mgjor contributors to this report are listed in 
appendix VTI. 

Sincerely yours. 




Linda G. Morra 
Director, Education 
and Employment Issues 
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Appendix I 

Federal Employment and Training Programs 
Proposed Funding Levels by Agency (Fiscal 
Year 1994)^ 



Dollars In millions 


Agency and programs 


1994 funding** 


Proorams included 
In analysis^ 


All programs (154) 


Total: $24,837.7 




Action - (3) programs 


Total: 100.9 




Literacy Corps 


5,3 


X 


Foster Grandparent Program 


66,4 




Senior Companion Program 


29,2 




Department ot Agricuiiure - program 


Total" 162 7 




rOOQ oiamp tmpioymeni & i raining 


1G2,7 


X 


Appaiacnian Hegtonai commission - u / program 


Total* 112 




Appaiacnian vocaiionai anu L^msr tiuucdiion rdoiiiuwo anu v^jjwiduuiio 


11,2 


X 


uepanmeni ot uommerco - {^j prugram* 


Total: 220.5 




Minority Business Development Centers 


24,4 


X 


American Indian Program 


1.9 




economic ueveiopmeni-varanis lor ruDiic vvurtvo anu uwvtjiuijniyiii 


135.4 


X 


economic ueveiopmeni-rUDiic woiks impdci rrugraiTi 


d 


X 


economic ueveiopmeni-ouppori lor ridnning wiydiiiiiduurib 


24,8 




economic ueveiopmeni-i ecnnicai Mssisidnce 


10.4 




economic UeveiOpmeni-OldlW dnu UULfdl euuiiunuu ljcvcjiujjuiomi rtaiiiiMiy 


4.5 


X 


Cr-ki!i/-»;oi t^/-^r\r\r\rrt\r* P^Qv/olnr^rnQnt QnH AHIiiotmont AcciQtflnr'fi PronrflnTi-^i iHHftn flnn 
oP"ddl eCyUnUiTlIU UyvyiUpiilWIU dllU /AUJUoUFIoIH /aooioicii H-fO • I'M'yiaMi wuuuoi 1 ai 

Severe Economic Dislocation and Long-Term Economic Deterioration 


19,1 


X 


Community Economic Adjustment 




X 


Department of Defense - (2) programs 


Total: 72.8 




Military Base Reuse Studies and Community Planning Assistance 


6,0 


X 


Transition Assistance Program 


66,8 


X 


Department of Education - (60) programs 


Total: 13,031 .4 




Even Start-State Educational Agencies 


88,8 


X 


Even Start-Migrant Education 


2,7 


X 


Women's Educational Equity 


2,0 


X 


Indian Education-Adult Education 


4,9 




Migrant Education-High School Equivalency Program 


8,1 




Migrant Education-College Assistance Migrant Program 


2.3 




School Dropout Demonstration Assistance 


37,7 




Adult Education-State Administered Basic Grant Program 


261,5 




Adult Education for the Homeless 


10,0 


X 


National Adult Education Discretionary Program 


9,3 




Vocational Education-Demonstration Projects for the Integration of Vocational and ' 
Academic Learning 


Vocational Education-Educational Programs for Federal Correctional Institutions _ ' 

' 1 J -n 
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Appendix I 

Federal Employment and Training Programs 
Proposed Funding Levels by Agency (Fiscal 
Year 1994)» 



Dollars in millions 



Agency and programs 


1994 funding*' 


Programs included 
in analysis^ 


Vocational Education-Comprehensive Career Guidance and Counseling 


,j 




Vocational Education-Blue Ribbon Vocational Educational Programs ' 


Vocational Education-Model Programs for Regional Training for Skilled Trades * 


Vocational Education-Business/Education/Labor Partnerships * 


Vocational Education-Tribally Controlled Postsecondary Vocational Institutions 


2,9 




Vocational Education-Tribal Economic Development ' 


Vocational Education-Basic State Programs 


717,5 




Vocational Education-State Programs and Activities 


81,3 


X 


Vocational Education-Single Parents, Displaced Homemakers, and Single Pregnant 
Women 


69,4 


X 


Vocational Education for Sex Equity 


31,1 


X 


Vocational Education-Programs for Criminal Offenders 


9,6 


X 


Vocational Education-Cooperative Demonstration 


f 


X 


Vocational Education-Indian and Hawaiian Natives 


15,1 




Vocational Education-Opportunities for Indians and Alaskan Natives * 


Vocational Education-Community Based Organizations 


11,8 


X 


Vocational Education-Bilingual Vocational Training 


0,0 




Vocational Education-Demonstration Centers for the Training of Dislocated Workers ' 


Vocational Education-Consumer and Homemaking Education 


0,0 




Vocational Education-TechPrep Education 


104,1 




National Workplace Literacy Program 


22,0 




English Literacy Program 


0,0 




Literacy for Incarcerated Adults 


5,1 




National Center for Deaf-Blind Youth and Adults 


6.7 




State Literacy Resource Centers 


7,9 




Student Literacy Corps 


6,1 


X 


Federal Pell Grant Program^ 


2.846,9 


X 


Guaranteed Student Loans^ 


5,889,0 


X 


Federal Supplemental Education Opportunity Grants^ 


125,0 


X 


Upward Bound 


160,5 


X 


Talent Search 


67,0 


X 


Federal Work Study Program^ 


89,6 


X 


Federal Perkins Loan Program-Federal Capital Contributions^ 


13,0 


X 


Grants to States for State Student Incentives 


0,0 


X 


Educational Opportunity Centers 


23,3 


X 


Higher Education-Veterans Education Outreach Program 


3,1 




Student Support Services 


110,3 


X 
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Appendix I 

Federal Employment and Training Programs 
Proposed Funding Levels by Agency (Fiscal 
Year 1994)' 



Dollars in millions 


Agency and programs 


1994 funding'' 


Programs included 
in analysis^ 


Postsecondary Education Programs for Persons with Disabilities 


8.8 




Rehabilitation Services Basic Support-Grants to States 


1.933.4 




Rehabilitation Services Basic Support-Grants for Indians 


6.4 




Rehabilitation Services Service Projects-Handicapped Migratory and Seasonal Farm 
Workers 


1.2 




Rehabilitation Services Service Projects-Special Projects and Demonstrations for 
Providing Vocational Rehabilitation Services to Individuals With Severe Disabilities 


19.9 




Rehabilitation Services Service Ptcjects-Supported Employment 


10.6 




Projects With Industry Programs 


21.6 




Supported Employment Services for Individuals with Severe Handicaps 


33.1 




Comprehensive Services for Independent Living 


15.8 




Library Literacy 


0.0 


X 


School to Work^ 


135.0 




Public Library Services * 


Department of Health and Human Services - (14) programs 


Total: 2,203.5 




Job Opportunities and Basic Skills Program 


825.0 


X 


Community Services Block Grant 


352.7 


X 


Community Services Block Grant-Discretionary Award 


39.7 


X 


Community Services Slock Grant Discretionary Awards-Demonstration Partnership 


4.4 


X 


Refugee and Entrant Assistance-Discretionary Grants 


12.6 




Refugee and Entrant Assistance-State Administered Programs 


84.4 




Refugee and ErUrant Assistance-Voluntary Agency Programs 


39.9 




Community Demonstration Grant Projects for Alcohol and Drug Abuse Treatment of ^ 
Homeless Individuals 


Family Support Centers Demonstration Program 


6.9 




State Legalization Impact Assistance Grants 


809.9 




Transitional Living for Runaway and Homeless Youth 


11.8 




Independent Living 


16.2 




Scholarships for Health Professions Students From Disadvantaged Backgrounds ^ 


Health Careers Opportunity Program 


i 


X 


Department of Housing and Urban Development - (4) programs 


Total: 303.4 




Emergency Shelter Grants Program 


51.4 




Supportive Housing Demonstration Program 


164.0 




Youthbuild' 


88.0 




Family Self-Sufficiency Program 


1 


X 


Department of the Interior - (2) programs 


Total: 20.9 




Indian Employ.iient Assistance 


16.9 


X 


Indian Grants-Economic Development 


4.0 
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Appendix I 

Federal Employment and Training Programs 
Proposed Funding Levels by Agency (Fiscal 
Year 1994)' 



• Dollars In millions 



Agency and programs 


lunaing 


Programs included 

m ariBiyoi9 


uepartment oi Laoor ** \ob; programs 






JTPA IIA Training Services for the Disadvantaged-Adult 


/yo. 1 


A 


JTPA IIA State Education Programs 




A 


JTPA IIA Incentive Grants 




Y 

A 


JTPA HA Training Programs for Older Individuals 


O 1 .o 


Y 

A 


J 1 rA iiu Ulsaavaniagea Youm 







JirA IIU Ulsaavaniagea Youin-inceniive laranis 


0*r.O 




J 1 r A IIU Ulsaavaniagea Youin-oiaie taucaiion rrograms 






J 1 rA lib 1 raining oervices lor me uisaavaniagea-oummer Youin cmpioymeni dna 
Training Program (Regular) 


1 fiftft ft 

1 ,\joo.o 


Y 
/a 


JTPA MB Summer Youth Employment and Training Program (Native American) 


k 


X 


JTPA EDWAA-Dislocated Workers (Substate Allotment)' 


229.5 




JTPA EDWAA-Dislocated Workers (Governor's Discretionary)' 


229.5 




JTPA EDWAA-Dislocated Workers (Secretary's Discretionary)' 


114.7 




JTPA Defense Conversion Adjustment Program 


m 




JTPA Defense Diversification 


n 




JTPA Clean Air Employment Transition Assistance 


0 




JTPA-Migrant and Seasonal Farmworkers 


78.3 


X 


JTPA-Employment and Training Research and Development Projects 


11.2 




JTPA Employment Services and Job Training-Pilot and Demonstration Programs 


35.1 


X 


JTPA-Native American Employment and Training Programs 


61.9 


X 


JTPA Job Corps 


1.153.7 


X 


Federal Bonding Program 


0.2 


X 


Senior Community Service Employment Program 


421.1 


X 


Apprenticeship Training 


17.2 




Trade Adjustment Assistance-Workers 


215.0 


X 


Targeted Jobs Tax Credit 


19.2 


X 


Employment Service-Wagner Peyser State Grants (7a) 


734.8 


X 


Employment Service-Wagner Peyser Governor's Discretionary Funds (7b) 


81.6 


X 


Labor Certification for Alien Workers 


58.6 




Interstate Job Bank 


1.9 




Youth Fair Chance^ 


25.0 




One-Stop Career Centers^ 


150.0 




Veterans Employment Program 


9.0 




Disabled Veterans Outreach Program 


84.0 


X 


Local Veterans Employment Representative Program 


77.9 




Homeless Veterans Reintegration Project^ ' 
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Appendix I 

Federal Employment and Training Progrants 
Proposed Funding Levels by Agency (Fiscal 
Year 1994)* 



Dollars in millions 


AgBnCy anQ prOyrainS 


iQQjt fiinritnn^ 

199^ lUIIMIIiy 


Programs Included 

III CIIICIiy9l9 


jou 1 laininy lor img norncicjss utJiriunouaiiun r lujyoi 




Y 
/\ 


UTTIC6 OT rorsonnGi ivianageiTioni ** \i / prograin 






reoBrai tmpioynrieni lor uisaovaniageo Touin-ourMmer 




Y 
A 


oiiiaii Dusiness AQrninisiraiion ** yy) prograrns 


TntAl* 1*^7 d 




mdndgcrncni ana ItJUnniUcil AbololanUtS lUr OUOlaHy anu COUilUIIMOany UlbaUVaiUaycjU 

Businesses 


ft 1 




Snnall Business Development Center 


67.0 




Women's Business Ownership Assistance 


1.5 




Veteran Entrepreneurial Training and Counseling 


0.4 




Service Corps of Retired Executives Association 


3.1 




Business Development Assistance to Small Business 


20.9 


X 


Procurement Assistance to Small Business 


33.7 




Minority Business Development 


22.7 


X 


Department of Transportation - (1) program 


Total: 1.5 




Human Resource Programs 


1.5 




Department of Veterans Affairs - (12) programs 


Total: 1,410.0 




All-Volunteer Force Educational Assistance 


895.1 




Selected Reserve Educational Assistance Program 


s 




Survivors and Dependents Educational Assistance 


109.1 




Vocational Rehabilitation for Disabled Veterans 


245.1 




Post-Vietnam Era Veterans Educational Assistance 


42.4 




Hostage Relief Act Program ^ 


Vocational Training for Certain Veterans Receiving VA Pension 


\ 


X 


Vocational and Educational Counseling for Servicemembers and Veterans 


u 




Service Members Occupational Conversion and Training 


64.5 




Health Care for Homeless Veterans 


28.3 




Domiciliary Care for Homeless Veterans 


23.4 




Housing and Urban Development/ Veterans Affairs-Supported Housing 


2.1 





•Programs identified are federally funded and provide for (1) assisting the unemployed. 

(2) creating employment, and (3) enhancing employabllity. The programs provide assistance to 

adults and out-of-school youth not enrolled in advanced-degree programs. 

^The proposed fiscal year 1994 funding amounts shov;n In appendix 1 are based primarily on the 
President's proposed budget, dated April 8, 1993. In those instances, when agency officials were 
able to provide us an estimate of the portion of the proposed budget that was used to provide 
assistance to adults and out-of-school youth, the amount has been adjusted. However, in other 
instances, when the portion of funds used for adults ^pd out-of-school youth could not be 
determined, the amount shown is for the entire program. 
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Appendix I 

Federal Employment and Training Programs 
> Proposed Funding Levels by Agency (Fiscal 
Year 1994)' 



'^Programs included in analysis were those identified as providing some assistance to the 
economically disadvantaged. 

^Economic Development-Public Works Impact: program funds included in Grants for Public 
Works and Development Facilities. 

•Community Economic Adjustment: funds allocated in 1993 are used to support programs in out 
years until funding Is depleted. 

'Data not available at this time. 

^Education loan program: amounts shown are estimates of loans for associate and nondegree 
programs, when possible to differentiate. 

'^School to Work: program proposed fo, fiscal year 1994. Funded at $270.0 million split evenly 
between the Departments of Education and Labor. Department of Education funding is from Carl 
Perkins Act: $15 million from National Programs-Research and Development and $120 million 
from Cooperative Demonstrations Program. Department of Labor funding is from the Job Training 
Partnership Act (JTPA). 

'Youthbuild: program proposed for fiscal year 1994. 

"Family Self-Sufficiency Program: job training, education, and support services are paid for by 
other programs, such as Job Opportunities and Basic Skills Training (JOBS) and JTPA. Federal 
funds may be used to cover local administrative costs. For fiscal year 1993, appropriations for 
operating subsidies permit the payment of $25.9 million to cover the administrative costs of 
operating the Family Self-Sufficiency program. 

•'JTPA IIB Summer Youth Employment and Training Program (Native American): funding included 
in JTPA IIB (Regular) program total. 

1*he actual funding for the JTPA Title 111 EDWAA program was increased significantly from the 
budget request dated April 8, 1993. The proposed funding for substate areas of $229.5 million 
was increased to $537 million. The proposed funding for the EDWAA Governor's Discretionary 
Fund v/as also $229.5 million, but was Increased to $357 million. Similarly, the Secretary's 
Discretionary funds were Increased from $1 147 million to $223 million. 

'"JTPA Defense Conversion Adjustment Program: funds allocated In 1991 used to support 
programs In out years until funding is depleted. 

"JTPA Defense Diversification: fun^s allocated in 1993 used to support programs in out years 
until funding is depleted. 

ojTPA Clean Air Employment Transition Assistance: no funds were appropriated for the Clean Air 
Act in fiscal year 1994. 

PNew program in 1994. 

'^The Homeless Veterans Reintegration Project was inadvertently omitted from our analysis of 
programs serving the economically disadvantaged. 

'Federal Employment for Disadvantaged Youth-Summer: program coordinated by Office of 
Personnel Management, but carried out by numerous federal agencies. Obligations devoted to 
administration not separately identifiable. 

•Selected Reserve Educational Assistance Program: funding included in All-Volunteer Force 
Educational Assistance total. 

^Hostage Relief Act Program: replaced by the Omnibus Diplomatic Security and Anti-Terrorist Act 
of 1986. No program funding used in any year, but available. 
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Appendix I 

Federal Employment and Training Programs 
Proposed Funding Levels by Agency (Fiscal 
Year 1994)* 



"Vocational and Educational Counseling for Servicemembers and Veterans: program funds 
included In other veterans programs, such as the All-Volunteer Force Educational Assistance 
Program. 
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Appendix II 

Outcome Data Collected by Federal 
Programs Providing Employment Assistance 
to the Economically Disadvantaged 



Agency and programs 


Participant 
employment status 


Participant skill 
attainment 


Participant wage 
levels 


Action 


Literacy Corps 


Department of Agriculture 


Food Stamp Employment & Training 








Appalachian Regional Commission 


Appalachian Vocational and Other Education Facilities and 
Operations 


a 


a 


a 


Department of Commerce 


Minority Business Development Centers 


a 


a 


a 


Economic Development-Grants for Public Works and 
Development 


Economic Development-Public Works Impact Program 


X 






Economic Development-State and Local Economic 
Development Planning 


a 


a 


a 


Special Economic Development and Adjustment Assistance 
Program-Sudden and Severe Economic Dislocation and 
Long-Term Economic Deterioration 


X 


a 


a 


Community Economic Adjustment 


X 


a 


X 


Department of Defense 


Transition Assistance Program 


a 


a 


a 


Military Base Reuse Studies and Community Planning 
Assistance 


a 


a 


a 


Department of Education 


Even Start-State Educational Agencies 


X 


X 




Even Start-Migrant Education 


X 


X 


X 


Women's Educational Equity 




X 




Adult Education for the Homeless 


X 


X 




Vocational Education-State Programs and Activities^ 


Vocational Education-Single Parents, Displaced^ 
Homemakers, and Single Pregnant Women 


Vocational Education for Sex Equity^ 


Vocational Education-Programs for Criminal Offenders^ 


Vocational Education-Cooperative Demonstration^ 


a 


a 


a 


Vocational Education-Community-Based Organizations^ 


Student Literacy Corps 


Federal Pell Grant Program 


Guaranteed Student Loans 


Federal Supplemental Education Opportunity Grants 


Upward Bound 




X 
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Appendix 11 

Outcome DaU Collected by Federal 
Progrmmfl Providing Employment AMlstance 
to the Economically Disadvantaged 



Agency and programs 


Participant 
employment status 


Participant skill 
attainment 


Participant wage 
levels 


Talent Search 


X 


X 




Federal Work Study Program 


Federal Perkins Loan Progrann-Federal Capital Contributions 


Grants to States for State Student Incentives 






X 


Educational Opportunity/ Centers 




X 




Student Support Services 




X 




Library Literacy ® 


Department of Health ond Human Services 


Job Opportunities and Basic Skills Program 


X 




X 


Community Services Block Grant 


Community Services Block Grant-Discretionary Award 


X 






Community Services Block Grant Discretionary 
Awards-Demonstration Partnership 


X 


X 


X 


Health Careers Opportunity Program 




X 




Department of Housing and Urban Development 


Family Self-Sufficiency Program 


X 


X 


X 


Department of the Interior 


Indian Employment Assistance 


X 






Department of Labor 


JTPA IIA Training Services for the Disadvantaged-Youth and 
Adult^ 


X 


X 


X 


JTPA IIA State Education Programs 


JTPA IIA Training Programs for Older Individuals 


X 






JTPA IIA Incentive Grants 


X 


X 


X 


JTPA MB Training Services for the Disadvantaged- Summer 
Youth Employment and Training Program (Regular) 




X 




JTPA IIB Summer Youth Employment and Training Program 
(Native American) 




X 




JTPA-Migrant and Seasonal Farmworkers 


X 


X 


X 


JTPA Employment Services and Job Training-Pilot and 
Demonstration Programs 


X 


X 


X 


Federal Bonding Program 


X 






JTPA-Native American Employment and Training Programs 


X 




X 


Senior Community Service Employment Program 


X 




X 


JTPA Job Corps 


X 


X 


X 


Trade Adjustment Assistance-Workers 


Targeted Jobs Tax Credit 


X 






Employment Service-Wagner Peyser State Grants (7a) 


X 






Employment Service-Wagner Peyser Governor's Discretionary 
Funds (7b) 


X 
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Appendix II 

Outcome Data Collected by Federal 
Programs Providing Employment Assistance 
to the Economically Disadvantaged 



Agency and programs 


Participant 
employment status 


Participant skill 
attainment 


Participant wage 
levels 


Disabled Veterans Outreach Program 


X 






Job Training for the Homeless Demons*- ation Project 


X 




X 


Office of Personnel Management 


Federal Employment for Disadvantaged Youth-Summer 


Small Business Administration 








Business Development Assistance to Small Business 


a 


a 


a 


Minority Business Development 


a 


a 


a 


Department of Veterans Affaire 


Vocational Training for Certain Veterans Receiving VA 
Pensions 


X 


X 





^Not applicable. 



'^The information shown in this appendix does not reflect the data collected by the National Center 
for Educational Statistics for the vocational education programs. 

^Starting In program year 1993, the JTPA Title IIA program was split into the Title IIA program for 
adults and the liC program for youth. Because our analysis began before the programs were 
split, the data for this appendix show the two programs as one program. 
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Appendix III 

Monitoring Activities by Federal Programs 
Providing Employment Assistance for the 
Economically Disadvantaged 



Agency and programs 


Compliance/ 
status* 


Financial 
activities^ 


Participant 
outcomes^ 


Action 


Literacy Corps 


X 






Department of Agricuiture 


Food Stamp Employment and Training 


X 


X 




Appalachian Regional Commission 


Appalachian Vocational and Other Education Facilities and Operations 


X 


X 


X 


Department of Commerce 


Minority Business Development Centers 


X 


X 




Economic Development-Grants for Public Works and Development 


X 






Economic Development-Public Works Impact Program 


X 






Economic Development-State and Local Economic Development 
Planning 


X 






Special Economic Development and Adjustment Assistance 
Program-Sudden and Severe Economic Dislocation and Long-Term 
Economic Deterioration 


X 


X 


X 


Community Economic Adjustment 


X 


X 


X 


Department of Defense 


Transition Assistance Program 


X 






Military Base Reuse Studies and Community Planning Assistance 


X 


X 




Department of Education 


Even Start-State Educational Agencies 


X 


X 




Even Start-Migrant Education 


X 






WomerVs Educational Equity 


X 


X 


X 


Adult Education for the Homeless 


X 






Vocational Education-State Programs and Activities 


X 


X 




Vocational Education-Single Parents, Displaced Homemakers. and 
Single Pregnant Women 


X 


X 




Vocational Education for Sex Equity 


X 


X 




Vocational Education-Programs for Criminal Offenders 


X 


X 




Vocational Education-Cooperative Demonstration 


X 


X 




Vocational Education-Community Based Organizations 


X 


X 




Student Literacy Corps 


X 






Federal Pell Grant Program 


X 


X 




Guaranteed Student Loans 


X 


X 




Federal Supplemental Education Opportunity Grants 


X 


X 




Upward Bound 


X 


X 


X 


Talent Search 


X 


X 


X 


Federal Work Study Program 


X 


X 
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Monitoring Activities by Federal Programs 
Providing Employment Assistance for the 
Economically Disadvantaged 



Agency and programs 


Compliance/ 
status* 


Financial 
activities^ 


Participant 
outcomes^ 


Federal Perkins Loan Program-Federal Capital Contributions 


X 


X 




Grants to States for State Student Incentives 


X 


X 




Educational Opportunity Centers 


X 


X 


X 


Student Support Services 


X 


X 




Library Literacy 


X 


X 




Department of Health and Human Services 


Job Opportunities and Basic Skills Program 


X 


X 


X 


Community Services Block Grant 


X 


X 




Community Services Block Grant-Discretionary Award 


X 


X 




Community Services Block Grant Discretionary Awards-Demonstration 
Partnership 


X 






Health Careers Opportunity Program 


X 


X 


X 


Department of Housing and Urban Development 


Family Self-Sufficiency Program 


X 






Department of the Interior 


Indian Employment Assistance 


X 


X 




Department of Labor 


JTPA IIA Training Services for the Disadvantaged-Youth and Adults^ 


X 


X 


X 


JTPA IIA State Education Programs 


X 


X 


X 


JTPA IIA Training Programs for Older Individuals 


X 


X 


X 


JTPA IIA Incentive Grants 


X 


X 


X 


JTPA IIB Training Services for the Disadvantaged-Summer Youth 
Employment and Training Program (Regular) 


X 


X 


X 


JTPA IIB Summer Youth Employment and Training Program (Native 
American) 


X 


X 


X 


JTPA-Migrant and Seasonal Farmworkers 


X 


X 


X 


JTPA Employment Services and Job Training-Pilot and Demonstration 
Programs 


X 


X 


X 


Federal Bonding Program 


X 


X 


X 


JTPA Native American Employment and Training Programs 


X 


X 


X 


Senior Community Service Employment Program 


X 


X 




JTPA Job Corps 


X 


X 


X 


Trade Adjustment Assistance-Workers 


X 






Targeted Jobs Tax Credit 


6 






Employment Service-Wagner Peyser State Grants (7a) 


X 






Employment Service-Wagner Peyser Governor's Discretionary Funds 
(7b) 


X 






Disabled Veterans Outreach Program 


X 


X 




Job Training for the Homeless Demonstration Project 


X 


X 


X 
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Monitoring Activities by Federal Programs 
Providing Employment Assistance for the 
Economically Disadvantaged 



Agency and programs 


Compilance/ 
status* 


Financial 
activities^ 


Participant 
outcomes^ 


Office of Personnel Management 


Federal Employment for Disadvantaged Youth-Summer ® 


Small Business Administration 


Business Development Assistance to Small Business 


X 






Minority Business Development 


X 


X 




Department of Veterans Affairs 


Vocational Training for Certain Veterans Receiving VA Pensions 


X 







•Monitoring activities concerning compliance and status include (1) reviewing the project to 
determine if it is meeting program requirements and following program procedures and 
(2) assessing the progress made in providing agreed-upon services. 



''Financial monitoring activities include determining if the project has (1) followed proper 
accounting practices and 0MB contracting procedures and (2) only spent funds on allowable 
items. 

^Participant outcome data include employment status, wages earned, and skills attained. 

^Starting in program year 1993, the JTPA Title llA program was split Into the Title llA program for 
adults and the IIC program for youth. Because our analysis began before the programs were 
split, the data for this appendix show the two programs as one program, 

^Program did not have a monitoring activity that was performed by federal officials. 
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Appendix IV 

GAO Identified Studies Published Between 
January 1990 and December 1993 on 
Employment Training Programs for the 



Economically Disadvantaged 




Program Participant Program 
Agencies, programs, and studies management outcomes effectiveness 


Action - Literacy Corps 


Development Associr;tes, Inc.: An Evaluation Report on the VISTA 
Litejacy Corps 


X X 


Development Associates, Inc.: An Evaluation Report on Volunteers in 
Service to America 


X X 


Department of Agriculture - Food Stamp Employment & Training 


Abt Associates, Inc.: Evaluation of the Food Stamp Employment and 
Training Program Final Report: Volume 1 


XXX 


Appalachian Regional Commission - Appalachian Vocational and Other Education Facilities and Operations 


Appalachian Regional Commission Office of the Inspector General: 
Audit Report for the Pennsylvania Welfare Reform Demonstration 
Project 


X X 


Tichenor and Eiche: Report on Review of Appalachian Regional 
Commission Work Force Excellence Initiative 


X 


Tichenor and Eiche: Report on Independent Audit of Appalachian 
Regional Commission Workplace Literacy Programs for the Southern 
Tier Centra! Region of Appalachian New York 


X 


M.D. Oppenheim and Company: A survey of the financial and 
programmatic records of the grants awarded by ARC to the 
Employment Opportunity Training Center of Northeastern Pennsylvania 


X 


Commonwealth of Pennsylvania, Department of Commerce: The 
Pennsylvania Self- Employment Demonstration Project 


X X 


Departtvy^nt of Commerce - Minority Business Development Centers 


U.S. Genera! Accounting Office: Minority Business: Minority Business 
Development Agency Needs to Address Program Weaknesses 


X 


U.S. General Accounting Office: MINORITY BUSINESS: Management 
Improvements Needed at Minority Business Development Agency 


X 


Department of Commerce - Economic Development - Grants for Public Worics and Development 


Department of Commerce - Economic Development-Public Works impact Program 


Mt. Auburn Associates: Evaluation of the U.S. Economic Development 
Administration's Public Works Program 


X 


Department of Commerce -Economic Development-State and Local Economic Development Planning 


Department of Commerce - Special Economic Development and Adjustment Assistance Program-Sudden and Severe 
Economic Dislocation and Long-Term Economic Deterioration 


Department of Commerce - Community Economic Adjustment 


Department of Commerce: Economic Development Administration Title 
IX Revolving Loan Fund Portfolio Status Report 


X X 


Department of Defense - Transition Assistance Program 


Department of Defense - Military Base Reuse Studies and Community Planning Assistance 
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6AO Identified Studies Published Between 
Jsnusry 1990 and December 1998 on 
Employment Training Programs for the 
Economically Disadvantaged 



Program 

Agencies, programs, and studies management 


Participant 
outcomes 


Program 
effectlvenees 


Department of Education - Even Start - State Educational Agencies 


Abt Associates, Inc., and RMC Research Corporation: National 
Evaluation of the Even Start Family Literacy Program 


X 


X 


X 


Department of Education - Even Start«Migrant Education 


Department of Education - Women's Educational Equity 


Department of Education - Adult Education for the Homeless 


U.S. General Accounting Office: Homelessness: McKinney Act 
Programs and Funding Through Rscal Year 1991 


X 






Department of Education - Vocational Education - State Programs and Activities 


Department of Education: National Assessment of Vocational 
Education Interim Report to Congress 


X 


X 




Department of Education Vocational Education - Single Parents, Displaced Homemakers, and Single Pregnant Women 


Department of Education - Vocational Education for Sex Equity 


Department of Education - Vocational Education - Programs for Criminal Offenders 


Department of Education - Vocational Education - Cooperative Demonstration 


Department of Education - Vocational Education - Community Based Organizations 


Department of Education - Student Literacy Corps 


Department of Education - Federal Pell Grant Program 


U.S. General Accounting Office: Student Financial Aid: Most 
Supplemental Educational Opportunity Grants Are Awarded to Needy 
Students 


X 






Department of Education: Study to Determine the Consequences of 
the Need Analysis Formula Contained in the Higher Education 
Amendments of 1992 


X 






National Computer Systems. Inc. and Macro Systems, Inc.: 
Department of Education. Office of Student Financial Assistance: Pell 
Grant End-of-Year Report 1990-91 


X 






Department of Education.- Guaranteed Student Loans 


U.S. General Accounting Office: Student Loans: Direct Loans Could 
Save Billions in First Five Years with Proper Implementation 


X 






U.S. General Accounting Office: Student Loans: Direct Loans Could 
Save Money and Simplify Program Administration 


X 






U.S. General Accounting Office: Student Loans: Characteristics of 
Defaulted Borrowers in the Stafford Student Loan Program 


X 






U.S. General Accounting Office: Testimony: Vulnerabilities in the 
Stafford Student Loan Program 


X 






U.S. General Accounting Office: Stafford Student Loans: Millions of 
Dollars in Loans Awarded to Ineligible Borrowers 


X 






U.S. General Accounting Office: Supplemental Student Loans: 
Legislative Changes Have Sharply Reduced Loan Value 


X 






U.S. General Accounting Office: Testimony: Student Aid Information 
and Private Tuition-Guarantee Programs 


X 
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GAO IdentUled Studies Pablithed Between 
January 1990 and Deeember 1993 on 
Employment Training Programs for the 
Economically Disadvantaged 



Program Participant Program 



Agencies, programs, and studies management outcomes effectiveness 



Department of Education: Trends in Guaranteed Student Loan and Pell 
Progrann Participation By Type of Institution 


X 






Department of Education: A Comparison of Projected and Actual 
Performance of the Verification Criteria In Three Award Years 


X 






Department of Education: Is There a Direct Relationship between 
Civilian Employment and Stafford Loan Volume 


. X 






Department of Education: FY 1991 Guaranteed Student Loan 
Programs Data Book 


X 






Department of Education - Federal Supplemental Education Opportunity Grants 


Department of Education - Upward Bound 


Department of Education - Talent Search 


Department of Education - Federal Work Study Program 


Department of Education - Federal Perkins Loan Program-Federal Capital Contributions 


U.S. General Accounting Office: Perkins Student Loans: Need for 
Better Controls Over Loans Recovered from Closed Schools 


X 






U.S. General Accounting Office: Perkins Student Loans: Options That 
Could Make the Program More Financially Independent 


X 






Department of Education: Correlates of Graduate Student Borrowing 
Patterns 


X 






Department of Education - Grants to States for State Student Incentives 


Department of Education - Educational Opportunity Centers 


Department of Education - Student Support Services 


Department of Education - Library Literacy 


Department of Health and Human Services - Job Opportunities and Basic Skills Program 


Manpower Demonstration Research Corporation: Effectiveness of 
California's Greater Avenues for Independence (GAIN) Program 




X 


X 


Renee S. Woodworth: The Promise of Jobs: Policies, Programs, and 
Possibilities 


X 






Congressional Research Service: Aid to Families With Dependent 
Children and Postsecondary Education 


X 






U.S. General Accounting Office: Welfare to Work: States Serve Least 
Job-Ready While Meeting JOBS Participation Rates 


X 






U.S. General Accounting Office: Welfare to Work: JOBS Participation 
Rate Data Unreliable for Assessing States' Performance 


X 






U.S. General Accounting Office: Welfare to Work: Effectiveness of 
Tribal JOBS Programs Unknown 


X 






U.S. General Accounting Office: Welfare to Work: States Begin JOBS, 
but Fiscal and Other Problems May Impede Their Progress 


X 






The Center on Budget and Policy Priorities, The Center for Law and 
Social Policy: JOBS In the South: A Review of Initial State Data 


X 






Department of Health and Human Services, Office of Inspector 
General: Review of On-The-Job Training Undor the JOBS Program. 
Ohio Department of Human Services 


X 


X 
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GAO Identified Studies Published Between 
January 1990 and December 1993 on 
Employment Training Programs for the 
Economically Disadvantaged 



Program 

Agencies, programs, and studies management 


Participant 
outcomes 


Program 
effectiveness 


Department of Health and Human Services - Community Services Block Grant 


Department of Health and Human Services - Community Services Block Grant 


- Discretionary Award 






Department of Health and Human Services, Office of Inspector 
General: Audit of Office of Community Services Discretionary Grants 
Awarded to Mexican American Unity Council, Inc San Antonio, Texas 




X 








Department of Health and Human Services, Office of Inspector 
General: Review of Discretionary Grants Awarded Under the Rural 
Housing and Rural Facilities Program 




X 








Department of Health and Human Services, Office of Inspector 
General: Audit of Office of Community Services Discretionary Grants 
Awarded to Mora Economic Self-Development Cooperative, Mora, 
New Mexico 




X 




X 




Department of Health and Human Services - Community Services Block Grant 
Partnership 


- Discretionary Awards - 


Demonstration 


Department of Health and Human Services. Administration for Children 
and Families: Demonstration Partnership Program Projects, 
Monograph Series 100-89: Case Management Family Intervention 
Models 




X 




X 


X 


Department of Health and Human Services, Administration for Children 
and Families: Demonstration Partnership Program Projects, 
Monograph Series 200-89: Micro-Business and Self-Employment 




X 




X 


X 


Department of Health and Human Services. Administration for Children 
and Families: Demonstration Partnership Program Projects. 
Monograph Series 300-89: Homeless Individuals and Families 




X 




X 


X 


Department of Health and Human Services, Administration for Children 
and Families: Demonstration Partnership Program Projects. 
Monograph Series 400-89: Early Prevention-High School Youth-At-Risk 




X 




X 


X 


Department of Health and Human Services - Health Careers Opportunity Program 


Department of Housing and Urban Development - Family Self-Sufflclency Program 


U.S. General Accounting Office: Public and Assisted Housing: Some 
Progress Made in Implementing HUD's Family Self-Sufficiency Program 




X 








Department of the Interior - Indian Employment Assistance 


Department of Labor - JTPA IIA Disadvantaged Youth and Adults* 


Avraham Lachs: The Effects of JTPA Approved Training on Earnings 








X 




U.S. General Accounting Office: Testimony: Amending the Job 
Training Partnership Act: Inadequate Oversight Among Issues that 
Need to Be Addressed 




X 








MGT of America. Inc.: An In-Depth Review and Evaluation of JTPA for 
the Alamo Service Delivery Area 




X 




X 




South Carolina State Council on Vocational and Technical Education: A 
Review of Two Years of Coordination: JTPA Programs, Secondarv 
Vocational Education, Technical Education 




X 




X 




U.S. General Accounting Office: Job Training Partnership Act: Youth 
Participant Characteristics, Services, and Outcomes 




X 




X 
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Appendix IV 

6AO Identified Studies Published Between 
January 1990 and December 1993 on 
Employment Training Programs for the 
Economically Disadvantaged 



Program Participant Program 
Agencies, programs, and studies management outcomes effectiveness 



San Luis Obispo County, Inc. Private Industry Council: Return on 
investment Report 


X 


X 


John Redman: Rural Development Perspectives: Federal Job Training 
for the Poor May Be More Cost Effective in Rural Areas 


X 


X 


National Commission for Employment Policy: Training Hispanics: 
Implications for the JTPA System 


X 




National Commission for Employment Policy: Evaluating JTPA 
Programs for Economically Disadvantaged Adults: A Case iStudy of 
Utah ana oenerai rinaings nesearcn nepon 


X 


X X 


U.S. General Accounting Office: Job Training Partnership Act: 
inaaeciuate uversigni Leaves J i r a vuineraoie lo wasie, MDUbe, ana 
Mismanagement 


X 




Abt Associates Inc.: The National JTPA Study: Title ll-A Impacts on 
Earnings and Employment at 18 Months 




X X 


U.S. General Accounting Office: Job Training Partnership Act: Actions 
Needed to Improve Participant Support Services 


X 


X 


U.S. General Accounting Office: Job Training Partnership Act: Racial 
and Gender Disparities in Services 


X 


X 


U.S. General Accounting Office: Testimony: The Job Training 
Partnership Act: Abuse of On-the-Job Training and Other Contracting 
is an Ongoing Problem 


X 


X 


U.S. General Accounting Office: Job Training Partnership Act: Services 
and Outcomes for Participants With Differing Needs 


X 


X 


National Association of Counties: The Challenge of Quality: Participant 
Selection, Recruitment and Assignment 


X 




Department of Labor - JTPA IIA State Education Programs 


John R. Petry, Memphis State University, and Fred K. Bellott. New 
Mexico State University: A Study of Terminees from JTPA Programs in 
Tennessee 




X 


Department of Labor - JTPA IIA Training Programs for Older Individuals 


U.S. General Accounting Office: Job Training Partnership Act: 
Information on Set-Aside Funding for Assistance to Older Workers 


X 




Jean Latting, University of Houston: Implementing Performance-Based 
Contracting in the JTPA Older Worker Program 


X 


X 


Department of Labor - JTPA IIA Incentive Grants 


SRI International: Effects of the 6 Percent Exemption Policy 


X 


X 


Department of Labor - JTPA IIB Training Services for the Disadvantaged - 
(Regular) 


Summer Youth Employment and Training Program 


Department of Labor, Office of Inspector General: Audit of 1992 JTPA 
Summer Youth Employment and Training Program 


X 




Department of Labor - JTPA IIB Summer Youth Employment and Training Program (Native American) 


Department of Labor • JTPA • Migrant and Seasonal Farmworkers 


Berkeley Planning Associates and SPR Associates: Evaluation of the 
JTPA Title IV MSf=Vv Program. Final Report 


X 


X 
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6A0 Identified Studies Publislied Between 
January 1990 and December 1993 on 
Employment Training Programa for the 
Economically Disadvantaged 



Program 

Agencies, programs, and studies management 


Participant 
outcomes 


Program 
effectiveness 


Department of Labor - JTPA Empioyment Services and Job Training - Pilot and Demonstration Programs 


Department of Labor - Federal Bonding Program 


Department of Labor - JTPA Native American Employment and Training Programs 


Department of Labor - Senior Community Service Empioyment Program 


Department of Labor - JTPA Job Corps 


Department of Labor - Trade Adjustment Assistance- Workers 


U.S. General Accounting Office: Dislocated Workers: Improvements 
Needed in Trade Adjustment Assistance Certification Process 


X 








Mathematica Policy Research, Inc.: International Trade and Worker 
Dislocation: Evaluation of the Trade Adjustment Assistance Program 


X 




X 


X 


Department of Labor. Office of the Inspector General: Audit of Program 
Outcomes in Nine Selected States. FY 1991 and 1992 


X 




X 




U.S. General Accounting Office: Dislocated Workers: Comparison of 
Assistance Programs 


X 




X 




Secretary of Labor: Study of Trade Adjustment Assistance Program 
Worker Certification Methods 


X 








Department of l^bor - Targeted Jobs Tax Credit 


U.S. General Accounting Office: Targeted Jobs Tax Credit: Employer 
Actions to Recruit, Hire, and Retain Eligible Workers Vary 


X 




X 




TVT Associates: Policy Evaluation and Review of the Targeted Jobs 
Tax Credit 


X 








Department of Labor: Targeted Jobs Tax Credit Program, State of 
Alabama 


X 




X 




Department of Labor - Empioyment Service - Wagner Peyser State Grants (7a) 


U.S. General Accounting Office: Employment Service: Improved 
Leadership Needed for Better Performance 


X 








National Commission for Employment Policy: Improving the 
Effectiveness of the Employment Service: Defining the Issues 


X 








W.E. Upjohn Institute for Employment Research: Labor Market 
Implications of ES Services for Duration of Joblessness, Probability of 
Subsequently Remaining Employed, and Repeated Spells of 
Joblessness: Comparisons of Ul Beneficiaries in Pittsburgh and 
Philadelphia. 1979-87 


X 




X 


X 


Department of Labor - Employment Service - Wagner Peyser Governor's Discretionary Funds (7b) 


Department of Labor - Disabied Veterans Outreach Program 


Department of Labor - Job Training for the Homeless Demonstration Project 


Department of Labor; Employment and Training for America's 
Homeless: Job Training for the Homeless Demonstration Program 


X 




X 




Paul A. Toro, Ph.D., and the Research Group on Homelessness. State 
University of New York at Buffalo: Final Evaluation Report: 
Demonstration Employment Project - Training and Housing (DEPTH) 


X 




X 




Cynthia D. Moehrlin. Elgin Community College: The Community 
College and the Homeless: A Model for the Nation 


X 




X 
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GAO Identified Studies Published Between 
January 1990 and December 1993 on 
Employment Training Programs for the 
Economically Disadvantaged 



Agencies, programs, and studies 


Progran^ 
management 


Participant 
outcomes 


Program 
effectiveness 


Office of Personnel Management - Federal Employment for Disadvantaged Youth-Summer 


Smalt Business Administration - Business Development Assistance to Smalt Business 


Small Business Administration - Minority Business Development 


National Academy of Public Administration; Assessment of Title II 
Demonstration Projects for Women Business Owners 


X 


X 




National Academy of Public Administration: Organization and 
Operation of the Minority Small Business and Capital Ownership 
Develooment Program: An Assessment of Progress under Revised 
Statutes P.L. 100-656 and P.L. 101-574 


X 


X 




Office of Inspector General, U.S. Small Business Administration: The 
7(j) Management and Technical Assistance Program of the Office of 
Minority Small Business Inspection Report 


X 






U.S. General Accounting Office: Small Business: Problems in 
Restructuring SBA's Minority Business Development Program 


X 






Department of Veterans Affairs - Vocational Training for Certain Veterans Receiving VA Pensions 




U.S. General Accounting Office: Vocational Rehabilitation: VA Needs 
to Emphasize Serving Veterans with Serious Employment Handicaps 


X 






U.S. General Accounting Office: Vocational Rehabilitation: Better VA 
Management Needed to Help Disabled Veterans Find Jobs 


X 


X 




Department of Veterans Affairs: Report of Survey 


X 






Bill Eddy: Three- Year Study of Significant Indicators Reflecting 
Outcomes and Performance of the Department of Veterans Affairs 
Vocational Rehabilitation Program 


X 


X 




Veterans Benefits Administration: A Report on the Vocational 
Rehabilitation Satisfaction Survey 




X 





^Starting in program year 1993, the JTPA Title IIA program was split into the Title HA program for 
adults and the IIC program for youth. Because our analysis began before the programs were 
split, the data for this appendix show the two programs as one program. 
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Studies That Evaluated Program 
Effectiveness: Methodology and Findings 



Title 


gain: Two Year Impacts in Six Counties — California's Greater Avenues for 




Independence Program 


Author and Date 


Manpower Demonstration Research Corporation, May 1993 


Program Purpose 


GAIN (Greater Avenues for Independence) is the California version of the 


JOBS program. The purpose of the program is to help afdc recipients enter 




the workforce and increase earnings and reduce welfare costs. 



Methodology The study included six California counties that accoimt for more than 

one-third of the state's gain caseload and more than one-half of its afdc 
caseload. A total of 33,000 afdc recipients, for whom gain participation 
was mandatory and who had attended an orientation, were randomly 
assigned to either an experimental group (who remained subject to gain's 
participation mandate) or a control group (who were precluded from gain 
but could seek access to other services in the community). The two 
groups* employment rates, average earnings, and average afdc payments, 
as well as the percentage of each group that left the afdc rolls, were 
compared during the follow-up period. Tlie differences between the two 
groups on these measures are the estimated impacts of gain. 



Among other findings, overall, about 29 percent of the single parents in the 
experimental group were working at the end of the second year, almost 
6 percentage points more than the control group (a statistically significant 
difference). About 51 percent of these single parents were employed at 
some time during the 2 years, compared with 45 percent of the control 
group. In addition, earnings for the single parents in the experimental 
group, for the second year of the study, was $2,712 per group member 
compared with $2,193 per control group member. This yielded an earnings 
gain, or impact, of $519 per group member (or 24 percent of the average 
control group member^s earnings). 

The proportion of single parents in the experimental group receiving any 
afdc payments had dropped to 61 percent by the end of the 2-year period. 
However, only a portion of this change can be attributed to gain since the 
control group experienced a similar decline. Nonetheless, three counties 
produced a statistically significant reduction in the proportion of single 
parents in the experimental group receiving welfare by the end of year 2. 
Similar results were also found for heads of two-parent families. 
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AppendLt: V 

Studies That Evaluated Program 
Effectiveness: Methodology and Findings 



Title 


The National JTPa Study: Title II-A Impacts on Earnings and Employment 




at 18 Months 


Author and Date 


Abt Associates, Inc., January 1993 


Program Purpose 


The purpose of the jtpa IIA program is to help the economically 


disadvantaged to compete in the workforce and reduce their dependency 




on welfare. 



Methodology in the national jtpa study, 20,601 jtpa applicants in 16 service delivery 

areas (sdas) across the country were randomly assigned to the treatment 
group (which was allowed access to the program) or the control group 
(which was not provided services) over the period November 1987 
through September 1989. The earnings and employment outcomes of both 
groups were then measured through follow-up surveys and administrative 
records obtained from state unemployment insurance (ui) agencies. Data 
on the baseline characteristics of the two groups were collected as part of 
the program intake process, and information about the employment and 
training services received was obtained from follow-up surveys and sda 
records. The study sites were not chosen to be representative of the nation 
in a statistical sense, but they did reflect the diversity of local programs 
and environments in jtpa 

This study gave estimates of the impact of jtpa Title II-A on the earnings 
and employment over the first 18 months after random assignment of four 
target groups — adult women and men (22 and older) and female and male 
out-of-school youths (16 to 21). 



Findings The results of the study are mixed. While jtpa Title II-A had generally 

positive effects on the earnings and employment of adults, adult men did 
not experience a statistically significant increase in earnings. The average 
18-month earnings of the adult women randomly assigned to the treatment 
group went up by an estimated $539, or 7.2 percent of the control gi^oup 
mean. Access to the program also increased the percentage of women 
employed at some time during the follow-up period by 2.1 percent. These 
estimates were found to be statistically significant and interpreted as 
reliable evidence of positive impacts on earnings. The average increase in 
the percentage employed for adult men was 2.8 percentage points and the 
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Studies That Evaluated Program 
Effectiveness: Methodology and Findings 



average earning gain $550 or 4.5 percent, similar to those for adult women, 
but it was not statistically signifi-^a^'^. 

In contrast to the findings fc e pro^^xam had little or no effect on 

the average earnings of feT^^ai. (a static Ucally insignificant earnings 

loss of $182 or -2.9 pci cent) and th^^ program actually reduced the 
earnings of male youths, on average, as evidenced by a large, statistically 
significant loss of $854 or -7.9 percent over the 18-month period. Access to 
JTPA had no significant effect on the 18-month employment rates of either 
female or male youths. 

Overall, the authors concluded that jtpa appears to have modest positive 
effects on the earnings and employment of adult men and women. But the 
program appears to have had virtually no effect on the earnings and 
employment of female youths and most male youths. In fact, it may have 
had a large negative impact on the earnings of those male youths who had 
been arrested before they applied to jtpa. 
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Studies That Evaluated Program 
Effectiveness: Methodology and Findings 



Title 


International Trade and Worker Dislocation: Evaluation of the Trade 




A(^ustment Assistance Program 


Author and Date 


Mathematica Policy Research, Inc., April 1993 


Program Purpose 


The purpose of the Trade Acyustment Assistance (taa) program is to assist 


workers dislocated by imports to reenter the workforce. 



Methodology The taa program offers Trade Readjustment Allowances (tras) and 

reemployment a(^ustment services to workers who lose their jobs due to 
increased import competition. In 1988, as one of several major changes, 
training was made an entitlement for eligible workers and tra recipients 
were required to participate in an approved training program. This 
evaluation describes the pre-layoff characteristics and post-layoff labor 
market experience of nationally representative samples of tra recipients 
who participated in the program either just before or just after the 1988 
program v^hanges. Data on ui exhaustees from manufacturing industries 
who did not receive tra are used for comparison purposes. 



Findings The study findings suggested that the training requirement reduced weeks 

of TRA receipt among the average recipient, despite the fact that the 
average duration of training increased. In addition, the training 
requirement led to a decline in the duration of initial joblessness and to an 
increase in earnings due to more rapid employment. 

However, the study concluded that whether training should be mandatory 
for TRA recipients should depend on how successful the training is in 
increasing employment and earnings. According to the study, the findings 
did not indicate that participating in training had a significant impact on 
the estimated employment and earnings differences of taa trainees and 
other TRA recipients. The study also did not find strong evidence that 
training had a substantial positive effect on employment and earnings, at 
least in the three years following the initial ui claim. Given the uncertainty 
about the returns of training the evaluation concludes that training should 
be voluntary rather than mandatory for tra recipients. 
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Effectiveness: Methodology and Findings 


Title 


National Evaluation of the Even Start Family Literacy Program 


Author and Date 


Abt Associates, Inc., and RMC Research Corporation, March 1993 



Program Purpose The purpose of the Even start program is to assist family members 

improve their literacy and, as a result, improve the educational 
opportunities of the children and the skills of the parents. 



Methodology The Even Start Family Literacy Program is intended to improve the 

educational opportunities of children and parents by integrating early 
childhood education for children with adult education for parents. One 
component of the evaluation provides in-depth information on a subset of 
10 selected grantees. At these sites, program participants were randomly 
assigned to treatment or control groups. This component of the evaluation 
focused on short-term outcomes of Even Start for parents and children 
and on the relationship between services received and outcomes. 
Short-term effects of Even Start were measured in four areas: (1) children, 
(2) parent literacy, (3) parenting skills, and (4) families. The results were 
mixed. 



Findings Even Start children gained sigxuficantly more school -readiness skills than 

the control group, but the results of two other methods used to assess the 
effects of Even Start on children showed no significant program impacts. 
In the area of parent literacy. Even Start showed a clear positive effect on 
GED attainment by program participants. However, measurements in two 
other areas, functional literacy levels on a reading test and reading and 
writing activities in the home, showed no significant program effects. Only 
one of four assessments of Even Start's effects on parenting skills was 
significant — the number of different reading materials in the home. Across 
measures of the effects of Even Start on participating families, including 
perceived social support and adequacy of financial resources, gains from 
program entry to the end of the first program year were minimal. 
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Title 


Labor Market Implications of es Services for Duration of Joblessness, 


Probability of Subsequently Remaining Employed, and Repeated Spells of 




TohlpQQTiPQQ' nnmni^ricinnQ nf ITT Rpnpfiriarips in Pittsburch and 




Philadelphia, 1979-87 


Author and Date 


Arnold Katz, Economics Department, University of Pittsburgh Preliminary 




Report for tlie Upjohn Institute of Employment Research 


Program Purpose 


The purpose of the Employment Service (es) is to assist individuals, 


including ui claimants, in finding jobs. 



Methodology The study, based on administrative data routinely produced by the 

Pennsylvania ui and es systems, examined the effectiveness of the es 
system in Pennsylvania. Detailed histories of work, unemployment, and es 
use of over 100,000 individuals, who registered with es between 1978 and 
1987, were assembled; their histories were then compared with the 
histories of a larger sample of nonregistrants. Pennsylvania was selected 
because (1) it has data dating back through a full business cycle, (2) it has 
a diverse economy, and (3) most important, it is the only state where the 
use of ES is voluntary for ui claimants. 



Findings The es had a positive effect on shortening the period of unemployment for 

UI claimants that were considered long-term unemployed, ui claimants 
who had been unemployed for 30 weeks or more returned to work 9 
weeks sooner th' they would have had they not used the ES. In 
comparison, IJI claimants who had been unemployed for roughly 12 weeks 
only reduced their unemployment, at most, by 2 weeks. The authors 
concluded that the shift from a 2-to-9 week reduction in unemployment 
. suggested that the ES is particularly effective in aiding a relatively small 
segment of the claimants who have trouble finding work on their own. 

Further, the study shows that most ES users accept jobs after exhausting ui 
benefits, suggesting that jobs obtained through the es are preferable to 
remaining jobless, but do not compare favorably with jobs held prior to 
becoming unemployed. Thus, the study concludes that the ES primarily 
acts as a backstop to prevent large earnings losses. 
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Title 


Evaluation of the Food Stamp Employment and Training Program 


Author and Date 


Abt Associates, Inc., June 1990 



Program Purpose The purpose of the Food Stamp Employment and Training Program is to 

improve food stamp recipients' ability to gain employment and increase 
earnings and to reduce their dependency on public assistance. 



Methodology The evaluation of this program was conducted during fiscal year 1988. The 

evaluation was based upon a classical experimental design involving the 
random assignment of about 13,000 eligible participants to either a 
treatment group, required to enroll in this program, or a control group, 
excluded from program participation. The evaluation was conducted in 53 
separate Food Stamp Agencies in 23 states. The sample was nationally 
representative of the different areas of the country in which the program 
operates, the types of food stamp recipients that participated in this 
program, and the modes of service delivery used in the program. 



The author concluded that the Food Stamp Employment and Training 
Program was found to have no effect on participants' employment and 
earnings and only a relatively small effect on the average food stamp 
benefits. In its first full year of operation, the program was not meeting its 
intended objectives of increasing participants' employment and earnings 
and decreasing their dependence on public assistance. 

Although program participants made substantial gains in employment in 
fiscal year 1988, the extent to which participation had an effect on 
employment must be derived from a comparison of their outcomes with 
those of the control group. The results of this comparison indicate that 
program participation in fiscal year 1988 had no discernible effect on 
participants' aggregate earnings, probability of finding work, amount of 
time worked, or average wages. By the end of the first year after the 
random assignment, over 50 percent of the program participants had some 
employment during the year; however, this gain is no different from that 
observed for the control group. The report also noted that the types of 
people participating in the program contributed to the questionable effect 
program participation had on their ability to find employment. Nearly 
70 percent did not have children and approximately one-half were single, 
highly mobile adults living alone. Most received no public assistance other 
than food stamps. Thus, the msyority of the program participants were 
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people who would have been looking for work regardless of the 
requirement to participate in the program. 

In terms of public assistance, the evaluation found an average $65 
reduction in food stamp benefits for program participants over the first 
year following certification for benefits— about 6 percent of the average 
annual total food stamp benefits paid to participant households. In 
general, individuals assigned to the treatment group received slightly 
smaller benefits per month and spent slightly less time receiving benefits. 
According to the study, these small differences probably reflected the 
program's effect on a small percentage of participants who either 
voluntarily left the Food Stamp Program sooner than they would have 
otherwise or had their benefits reduced or terminated for noncompliance. 
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Title 


Summary of Final Evaluation Findings From Fiscal Year 1989: 




Demonstration Partnership Program Projects. Monograph Series 100-89: 




Case Management Family Intervention Models 


Author and Date 


U.S. Department of Health and Human Services, Administration for 




Children and Families, Office of Community Services, June 1992 



Program Purpose The purpose of the Community Service Block Grant demonstration 

projects is to study ways to assist people on welfare to become more 
self-sufficient. 



Methodology and Findings This study evaluated five case management self-sufficiency initiatives. The 

initiatives involved multiple agencies joining together to provide 
coordinated services to individuals or families requiring public assistance. 
Use of these services is brokered or directed by a case manager who is 
responsible for coordinating the care received. Four of the five evaluations 
used either a randomized or constructed control group methodology to 
assess the impact of the initiatives. The results from the four evaluations 
using a comparative analysis were mixed: 

P'amily Self-Sufficiency Project : An analysis of afdc data comparing the 
study group with two comparison groups across 15 months (April 1990 to 
June 1991) showed no statistical differences in the average amount of 
assistance received by the families in the three groups. Although the 
proportion of families receiving assistance and the average grant amounts 
for the three groups decreased across time, they decreased equally. 
However, the study concludes that the program clearly was able to 
improve parents* self-esteem and problem-solving skills, as well as the 
quality of their social relationships. 

Self-Sufficiency Plus : The only documented outcome of the program was 
in the area of education. The outcome effects for education were small, 
but they consistently pointed towards greater educational attainment by. 
those participants who received all of the program*s services. The study 
assumes that the educational advantages achieved by the treatment group 
are precursors of later increased employment and wages, and concludes 
that additional tracking of the participants would be necessary to 
document the ultimate impact of the program. 
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Operation Comjmunity Uplift : For two experimental sites, the study 
concluded that on-site interventions were a significant factor in increasing 
total family income and nonpublic assistance income. Interestingly, more 
experimental households increased their public assistance income. This 
outcome contrasted with the self-sufficiency goals of the project, but the 
study indicates this is a short-term solution to immediate problems, 
making it possible for people to work on longer term self-sufficiency goals. 

Project HOPE-Headstart Opportunities for Parents Through Employment : 
An evaluation conducted at the project operation's midpoint shows the 
following: a slightly higher percentage of control group members were 
employed, but a significantly larger percentage of experimental group 
members were involved in educational and training activities that had the 
potential for increasing long-term employment opportunities. In addition, 
the unemployment rate (no job or training involvement) of parents in the 
experimental group was one-third of that in the control group. A 
subsequent evaluation attempted at project completion had low response 
rates, which made it difficult to draw statistical conclusions regarding the 
programs's success. However, based on the limited response, the study 
concludes that the HOPE project was not totally successful in its goal of 
having all participating parents employed by the end of the project. There 
were significantly more employed parents in the control group than in the 
experimental group. But the analysis also showed a significant number of 
parents in the experimental group involved in activities that had the 
potential for rewarding careers. Consequently, the study classified the 
project as a success. 
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Title 



Summary of Final Evaluation Findings From Fiscal Year 1989: 
Demonstration Partnership Program Projects. Monograph Series 200-89: 
Micro-Business aiid Self-Employment 



Author and Date 



U.S. Depaitment of Health and Human Services, Administration for 
Children and Families, Office of Conununity Services, June 1992 



Program Purpose 



The purpose of the Community Service Block Grant demonstration 
projects is to study ways to assist people on welfare to become more 
self-sufficient. 



Methodology and Findings 



These evaluations present the results of four programs designed to help 
low-income people achieve economic self-sufficiency by starting their own 
businesses or pursuing self-employment. These evaluations used either 
random or constructed control groups with which to make comparisons 
with the groups receiving services. On the basis of their analyses, the 
authors of each of these evaluations concluded that the programs were 
generally successful. 

Operation INC (Incubator for New Companies) : Data were collected on 
the frequency of biosiness start-ups or funding for both participants and a 
control group. During the 14-month evaluation period, 38 percent of those 
in the experimental group were funded for a business compared with 
33 percent in the control group. Of the 15 businesses started by the 
experimental group, only 1 was unsuccessful. According to the study, the 
experimental group*s business, with its success rate of 93.4 percent, far 
exceeds any national success rate in the general population. 

Partners in Progress : After 24 months, both the participant group and the 
control group were compared and evaluated. According to the study, case 
management services improved employability over a 2-year period. 
Women who received vocational training along with case management 
services were more likely to obtain employment than those who did not 
receive these services, although the hourly rate is not higher. Low-income 
women who received entrepreneurial training and case management 
services were more likely to have taken steps toward self-employment 
than low-income women who did not receive these services. 
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Capital Opportunities : Preliminary results of the project show that 
93 percent of the program participants pursued and expanded their 
business ventures, compared with 50 percent of the control group. The 
receipt of a loan appeared to increase participants' sales and 
proportionately increase their personal salaries. On average, the loan 
recipient received less welfare assistance per month than the comparison 
group, $178 versus $208. Even though the report's evaluation only 
provided a preliminary assessment of the project, the study concludes that 
the program aided low-income people in pursuing or continuing their 
business ventures. 

Bright Center Demonstration Partnership Project : At program exit, only 
16 percent of Bright Center participants were imemployed, compared with 
26 percent of the control group. At the i3-week follow-up. Bright Center 
participants who were unemployed decreased to 13 percent, while the rate 
for comparison group members increased to 36 percent. According to the 
study, an examination of the outcomes for Bright Center participants 
indicate the program is successful in providing training and support 
services to low-income women assisting them to achieve self-sufficiency. 
An overview of outcome data supports this conclusion, although the study 
indicates that the small number of control group members made a 
statistically sound comparison impossible. 
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Title Summary of Final Evaluation Findings from Fiscal Year 1989: 

Demonstration Partnership Program Projects. Monograph Series 300-89: 
Homeless Individuals and Families 



Author and Date U.S. Department of Health and Human Services, Administration for 

Children and Families, Office of Community Services, June 1992 



The purpose of the Community Service Block Grant demonstration 
projects is to study ways to assist people on welfare to become more 
self-sufficient. 



Methodology and Findings This study evaluated three projects designed to increase the 

self-sufficiency of the homeless. Two of the projects used control groups 
to control for potentially confounding variables, mainly demographic 
differences. 

Homeless Family Self-Sufficiency Project : The evaluation compared 
families with little or moderate case management with families who 
received intensive case management Among the findings, those families 
who received high case management were significantly more likely to 
move from a condition of no income or dependence on afdc benefits to a 
condition of supporting themselves through a combination of employment 
and benefits. At the time of the evaluation, few families were able to 
support themselves tlirough employment alone. However, families who 
received high levels of case management also appeared to exhibit greater 
positive change in their housing situations. The study concludes that most 
families make positive moves toward self-sufficiency when given some 
support in the form of case management. 

Homeless Employment Partnership : An experimental design was used to 
evaluate the impacts of the project. Even with the control group having job 
referral assistance and job search resources at their disposal, the case 
management clients did much better on identified indicators of 
self-sufficiency. The odds of having a job in the case management group 
was almost four times as high as in the control group, even when school 
status, race, barriers to employment, and past treatment were held 
constant. Key job benefits, such as health insurance and sick leave, were 
higher for the case management group than the control group. They were 
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also three times less likely to be homeless. The study concludes that the 
project was successful. 
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Title 


Summary of Final Evaluation Findings From Fiscal Year 1989: 




Demonstration Partnership Program Projects. Monograph Series 400-89: 




P^.arlv Prpvpntinn-T-TioH RpHnnl Vr»nfVi-at-T?iclr 


Author and Date 


U.S. Department of Health and Human Services, Administration for 




Children and Families, Office of Community Services, June 1992 


Program Purpose 


The purpose of the Community Service Block Grant demonstration 




projects is to study ways to assist individuals on welfare to become more 




self-sufficient. 



Methodology and Findings This study evaluated two projects designed to increase self-esteem, 

improve school performance, and increase the job search and 
performance skills of targeted youth. Both projects used 
quasi-experimental designs in their evaluations. In both projects, local 
community action agencies and the targeted high school were the primary 
agencies. Both programs used as their primary intervention a specialized 
course, designed to improve self-esteem and decision-making skills. 
Although attrition in the control groups made analysis difficult, both 
evaluations concluded that during the Ufe of the project, the experimental 
groups had more improvement in all categories than the control group. 

Partnership for Youth Self-Sufficiency : The results of the evaluation are 
mixed. Among other findings, on average, treatment group students were 
employed more hours per week than comparison group students. Among 
AFDC students, those assigned to the treatment group began with fewer 
hours per week, but surpassed the afdc students in the control group 
halfway through the fii-st intervention semester. The evaluation also shows 
that students in the treatment group had a greater knowledge of 
preemployment skills than the students in the control group. However, the 
average wage per hour actually declined for both groups of students over 
the tracking periods. No significant gains in grade-point average were 
shown by either the treatment group or control group. In addition, the 
evaluation did not detect any differences in dropout rates between 
students in the treatment and control groups in the initial stage of the 
project. However, later refinements to the intervention may have remedied 
this condition, as early data from the second phase suggest. The study 
concludes that the service model developed and implemented had 
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potential for future adaptation and replication with comparable high-risk 
populations. 

High-Risk Youth Demonstration Project: It was expected that the 
interventions instituted by this project would lead to higher self-esteem, 
higher academic achievement, and greater success in the labor market. 
The study concludes that these expectations were clearly met. Although 
not always statistically significant, experimental group students, as 
compared with the control group, showed greater gains over the project 
period in scales used to reflect changes in self-esteem. The experimental 
group also showed greater increases in grade-point average, attendance, 
units completed, and graduation rate. In addition, the experimental group 
had a dramatically lower dropout rate, and had more success in the labor 
market. 
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Title 


Evaluating jtpa Programs for Economically Disadvantaged Adtdts: A Case 




Study of Utah and General Findings Research Report 


Author and Date 


National Commission for Employment Policy, June 1993 


Program Purpose 


The purpose of the program is to assist the economically disadvantaged to 




find jobs. 



Methodology and Findings This study compared Utah JTPA II-A adult enrollees in program year 1987 to 

a comparison group which was constructed using various techniques fi-om 
a random sample of ES registrants who received basic employment 
assistance during program year 1987. Net impacts of the jtpa II-A adult 
programs were estimated using non-experimental techniques, which 
compared the observed outcomes of program participants with those of 
the ES comparison group. 



The employment impact estimates for Utah's Title II-A adult enrollees for 
program year 1987 suggest jtpa participation has a strong positive effect 
on employment for adult men and women who complete more intensive 
training programs and who are placed into fidly unsubsidized private 
sector jobs. Participants who completed training and were placed in 
unsubsidized jobs (or were retained by their employers in the case of 
on-the-job training) had a significantly higher likelihood of being employed 
2 years after their original program enrollment. Both adult women and 
men were roughly 10 percent more likely to be employed. In addition, jtpa 
resulted in higher second year earnings, if employed, for those placed 
through more intensive training programs. For adult women, this earnings 
impact is stronger for those enrolled in on-the-job training, while for adult 
men it is stronger for those enrolled in classroom training. 
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See comment 1 , 




OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE 
33O0 DEFENSE PENTAGON 
WASHINGTON, DC 20301-3300 



2 2 FEB TO 




CCO»*OMIC MCUIt'TV 

Ms. Linda G. Morra 
Director 

Education euid Employment Issues 
Health Education and Hiiinan Services 

Division 
U.S. General Accounting Office 
Washington, DC 20548 

Dear Ms. Morra: 

This is the Department of Defense (DoD) response to the 
General Accounting Office (GAO) draft report, "MULTIPLE 
EMPLOYMENT TRAINING PROGRAMS: Most Federal Agencies Do Not Know 
If Their Programs Are Working Effectively," dated February 3, 
1994 (GAO Code 205241/OSD Case 9512). The DoD has no comment on 
the information contained in the report, but disagrees with the 
inclusion of the Military Base Reuse Studies and Community 
Planning Assistance Program. 

According to the draft report, the GAO focus was on Federal 
programs that provide employment training assistance to adults 
and out of school youth. The Military Base Reuse Studies and 
Community Planning Assistance Program, however, does not directly 
address employment opportunities or training for individuals. 
Rather, the program doea what the title indicates — it helps 
States and local governments fiian for reuse of closing military 
installations. Planning for reuse of the .base involves the 
community deciding the best mix of industrial, commercial, 
aviation or residential, public or private use for the property 
that will best suit the needs of the community for economic 
development and jobs or public facilities and open space. The 
DoD tracks employment at a sample of former military installa- 
tions as means of estimating the redevelopment and new job 
creation experiences of communities with former military bases. 

The Department appreciates the opportunity to comment on the 
draft report. 



Sincerely 




il J. Dempsey 
Director 
Office of Economic Adjustment 
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Humtn Setourcfli Diviikm 
United Stftiet Omni AecouBting Office 
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ManhaU S. Smith 



Asatbment 
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See comment 3. 
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Z>iltv€ry ofS^nku (OAO/HEHS-94-71, January 34, 1994) and MtuXtipU tit^lpymtni Tyaiwi^ 
Progrmt: Ovtrit^ping ProgramM Can Add Unn9c$ssary Costs (GAO/REBS-94-S0. Jamury 2S» 
1994). 

The draft t«p«rt docs not rctkct the 
coUsedOB 9i Mtconis dste on 



s Wdntirei to sddNsa dcitdcackt in tbc 
ia procrm rtodiing Mml sdncitloa ftiodf . 



The DeputttMOi of Bdwjrion reoocnizai ttei dettiiod omoomea dm for eo^iloymtsc trtioinc 
pfognm«ienotcanf«ttlty«vtitab^ Ch a fWyn n&l exkt wiih the 

ooUtcdon, aflslyui, ud npoKtBg of this 4itft< Howercr, (he Department it makiof m^ 
amdos towtrd foretihiit this tMdd. 

The Atehifatzvtta's sdMstSoD rtfora sMW 

ivhidi art anmdy pmUDi btfm Conl^raoi, IwMsi sffocte to vopport ^ dsralopnMrt of 
j^fafMSBeo] 



fhnh TflOft- TMnritfi A-^i ^^ii^j^p* ^ An«tnp ^^iw^r. arvi 

ihiUs stMdada oooiirtrat widt ariipiud aUntodt. This legtaUtUm wm enoourtte tuiM 
and local to icvarap jhek cotoe edscation tysma, psitkulariy duough 

xttormM of cucrkuls and atalT, lo fnaMe atudcota to meet high srsrtwnk. itsodscda aad tt> 
movetowirda&oulcon»socoa<4hUHy lyi^ Moreovsr. k will Miiri>Bah a nsiksal 
lyaiem «f occ^pslioMl ahiU itaaiiidi bcKhmailDDd to wgrki class levels. 



Oosk 2000 pcovidas dts waptaa jCbr ewo odter pisoea of adirjtion it&)im kgialation: 
ImptoviDf Anvrks'i Schoola Act] of 1993 and iba SchooHo-Work Oppoctu&Uka Act of 
1993. 

The T'n^^fttfb** ttrtvw*!^ yj}^ , which rtscflwrncs the fedsral 

ttK s mm s BA 'a pregiasBS in ekoastaty and a s<Ywtafy sducatkm, rtinfiifcea Ciosla 2000 
by *'*'^««"f acoeas to quality edkadon. It taxfsca ftdsral imnoaitt in sdiicatioa 
toiwiid diiidviiiaged chSdicn and youtfi-^soxfurating die adopcioa of school-wide 
^jproacta in high-poverty acbooiiHo caiwe diat diBy hl^6 d^ 
y>-f <i ^ p ii *t alandazda. 

Tl^ SAool-itvWQrk Oan rt rnmHii a joint initiBtire of «m9 Dspftrtmenia of Bduca^ 
and Labor pcooKMUdiacratkm of conpittwoiive lyitama in evaty ataia cod local 
cocmnity to provlds a amooth, iktt ciiaUeafhig, tracsiiiozi frtsn acfaool to cateer 
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opportunities for youti not goin^ pa to four-yc«r poatiecondtry iMtitaaons. Tte 
9ctx)ol-to<Woris leipilMion coabiHn academic and occupadom skills, work experience, 
ttri a ^jfHtytig curriculum articbliting bigb school aad two years of coUece «1vsre 
ifproprisM, or iddWomt fmhi^g: A stodeiM wfu> completM a schoot-to-work pros^ 
wovil receive boA a high 9chooi ^Iocqa and a 3]aUi certiflcatB bcncfamarkcd xo 
r^tle^'ng ncfi« %wie iikI sJcilli stiodards that win f>e recosoised M&»wide. 

WItUM the DcvvttMat, tftorts «re underwey to stnogtlMi ateteiitraOve stivctana fer 
alp^Mif amcaMS data eo ov Most hnkKiraBt procranc 

• IHMnrmr F^HTtfim iWrtnH Tto wiaier, stratesfc plam were developed « 
oe5c6 level for every protraffltW support offiee ia 6e Depanm^ 
DepaitmiAleytf toacottcU^aoadplaii. Tbepurpoieof tUi exiSDihwi^aaaiag ieto 
idcntifrlBey priorities at flic ofiScj and Depart^ 
mauotettMitt syslims to track tfajperfonnancc of the De paihu etfs pcogrvas. 

ij^bocaisriseitttatenmMaaiivaia^ UDdsrdOs kbonrtocy, each of ED'e 17 
ariacipal qpmtfe« c?<inip9'*^ «*'***'^^fK^ peiteri n aflca aieasurae foe <is most haavi^y 
bMdgcMproncMaadcKMcritiklUn^iioas. Hie Dipartmeat was also sdecled as a 

piias»aieri>eOowew^^ 

aO activkks within dvB DeUtoM tte am reM 
snsteM fiMcU aU programs, ax^ to report on the DcpartDxot'^ 
hi«h Inrela of perforttsnoe propoijed la the pirn. 

hi aiiMsw. fht irTT^^* ' * — j^mU^^ jU^iimimi tMrftmanca iiaawta 

NrMaSMUer pwpwni. The fbOcriHi^ ejffiM hive hesa untertak^ 

develop parfetaunce mMSuxea or iodk^oib of program quaUty far ntfor pvognm areas. 

• AAifrf ^Hnt iiKNaaooalUleraoyActof 199lTeqphea<»devetopoQam^ 

■■iltiaiirs nf f rnimn iiir'^ ^ — ■ ^ i^ ^ftmnc^ 

iin^w Agt taciaiing Mgaaa M iaueHBa w t aod i li a iHiM ef i^a^ases wm4 

tearvvwntiattellsmeyskfilBariSttdcBta. the Ptpartmf nt, fa canaoltatiott whfa 

MM. experts in die field. aM edkicam 

^pitHly Worksbopehxvebeeacob^facSBdlcr snsadkaclocf oTaM^ 

i^e^ihle fcr dcvelopici and wbtacntifli stale ifldfeatois of proystt tpeltey. 

« y^g^JI^MIte The Cad p. PerUsiVocMiQCMl aiid Applied Teeho^^ 
BdaoaioA AiDt (VMisa AcO laqolm sissss 10 devekv and kipata^ 
MMna sbA stntfaids for ssaasili^ ilM perftcnattM of secoodscy lad poetseeoodaiy 

iresaiaMi srtwrtlea prt >lz a miahmiffl, each state nist taiMa at 

iseurm fa iWr wfff ttj^frt^ *y^i «^ of which most be ai faidteaiof of kanin 
aad cametaD(7 gakia, with the iMher meaMuaa befac ettec LuuiiiSiag atflalnMi^, 
Jefc ar WW* Ail attatamart, c-fplatta ef eac^ 

If* - ' ^*-*^f ..M^ ^ timmamw^M^ 
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The Departoient has provided tecimical assistance to states aod has conducfiod workshops 
oa the iiupknicntation of state perfonnaace meisures systcnn. 

• Vocarioial Rehabilftation. The 199Z AmeixScDents to me HehalnUtatioxi Act is^Mc a 
requirAcnem to dtv«k>pmem and iinpkmtncatioii of cvthiatkm. icuidatdi anl 
pertontnce tndicaiori for sh0 htdhi $\m gnat prognm. Tlw SihibiUfKioa Svvkm 
Admixiiitrtdoa it direcied 10 rtquirt ifaac nMM report a&aualiy on ibdr con^liance wiih 
the standards. The Department hat established a KegiUtDoitt Policy Qioup to work with 
expert! to develop tegulatikxis to it pabUahed by September 1994. 

The DciMriaietit of Educfttfam hee eUo qude major aMke hi evabaedof the effecti^fsco of 
minyoTtlieproinimeldMidnedfatfaedNflre^ The 
Deptrtmcit^f noM-yeer t!valiitttoa p^ 

idtatitf dk^eXtntdntMU^ ' 1 crni^ni numrirr firi MiToiii ■■liLiiiiiiii 

erehiettoQe ere uftdonmy to eTehietfon profram inpect, iadudhic to exaiqple.ifae foUowiog: 

• Start Prffgnm The PcpertyKot entoaSed flic inytemrnfttioo aod iagHct of 
prpiects funded under Ail piognnti, ptOYi^ 

servicea, coocdtedoQ. iaqttemeftta^ 1989 
aad FT 1990, as well as an iD-dofdi, raadom aaaignpifiit aaeefimeac m aelect sitec. The 
1^ report wu subcBitlDd to CoD^^ in Sqvle^^ 

• A AiH Pii»Mri/ffl Fiwam 111 ryj 1990. the Dqiertmett Ittncted a luitk^ 

study of adult c docackm pcotrrot; and peitiripeatfi. bi addxtioa to colleetmg mfonnaiksai 
od lervke providen, clients exiies|us ^ protcam 0m a OM-yeer period were followed 
for up to 18 moQlhi to obtaia coifiivdiexiiive meMwxei of the imanjiity aod duxatioii of 
pertidpatkxL lite nml lepoct wiji be aTaikUe in 1994. Addxtiooel aoalyeei win be 
condocled on reciuitiDent, progrtia fetatttku, learah^ stiai, and dianfei hi ea^loyxoent 
status. 

• School DroyntiyiiionatiitmAiitt^^ PeofrpU' The Deptrtmefit's correct 
demonatratiopinsderthiaptogramiiacecayedoflCTW reatnicturing 
aod tarieted htferveodeoi dSrecad spedflcaOy toward Hadiffii at riak of droppmf out. 
An ao-d^Hh evataiatkm of seteciad projecta ia •»*^«i«fa^ Utt^t^rMtmym of iheee 
profpracoe. A nuodom anipneitt|evahiaiioQ of ttddeot o» i c? Q<nea va utpmA projects wu 
begun m the 1992^ adiool yetri the final year of data coCectvn win be the 1994-95 
echool yeei. 

• Natkaal Worfcnti^ lim ^ Ptofram. This evah»tkm wiU take a detailed look at 
prq)ecli ftanded under mia program in FY 1993. Using a randoci igmnaiC design 
where feecihle^ ihie atkdy will iiajMa project impect en worker job pecfbisunce, 
pnOocMy, reieoikn, end eemidtt. Through caae itodiea of these projects, effective 
workplace liseracy piactkree will M ideadfled. 

• Kanonal Aaee iimenrof Vnr^t^ftn^i ^^^tkn fNAV^ Required by aectkn 403 of (he 
Parkma Act, NAVB b specifically charged with reporthig on the acadeimc and 
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cmployDOWOC ouicoo« of vocaiioaal tiducation. NAV'E wiU report on outcoc^^ i»f 
iccoaiary lu! powecoodiry vocjidonil cducatioa. inchicliag enaployiawt and etmings. 
NAVB's interim reponwM*\ilimiticd to Cong^ 1993 and the tSnai 

report wjll be mlmitled oc or before July !, 1994. 

• p^t ^i^T^tnr RvilinThH ^Tf^* ' Ev«tt thougji au)« poitMcooitfy pfOfrtntt do ^ 
contain faa^ set-aiklM to piy fiv ervaluatknu, dae Deptnment's effocti bave grown 
jignjficamly in recent year*. Curient student Ud *valoatioo« inchide potsibilities for 
measurtng: the cotcoim of propriiary sdwols, analyziog tte elftcts of student aid on 
enrolibient deciikMU. emninir^ die Mn burdea of recem college gfaduatea. and 
KDjOyzing factors related » loan d^dailts and die effiKtivenMa of meaairoi to reduce 
de&Kilts. ThcDepourtDmtbasKecj&aaytiecQnAns^ 
Loan Program. 

In addiCk>dtoprofidixeiDfomaltaoa|tbeB4v<>^ 
I)n«tiiiwt to tapcvve Uw ccDectlM of j 
ftd«»l edwadk. fimda, i»e dbq h«n oA^^ 
the drtfl nport 1^ tltt mwser fa iiUA tt afft^ 

repert. 

Illmj uf llMi iw iiftTi^ iai Indirt fn Tlir Tfrrfr ttt^ ^ ' TT"" "''^ *^ «wfhtttfii«> 

of "fnpkBjwmt tnUif progiw**** 

The term "en^>k>yii»nttnfflmgpco^ *diote profxami and icleied ftindiAf 

stzeami that provide euyloyoieec vatnfaig io adulB aod out-of-Kbool youthi* m foottote 2 to tbe 
draft OAO report Wedi»M;reediainiiiwofd»pw«f«madiacaaiadln1^ 
defmition. OAO't initial <!e«enoawtion a*{^ 

^jpira^y f eOecnd in MiUifpU Bipiaymikt Progrmu: NaUcmU Brpipymint Tramini Sina<tfy 
//MM (QACVTOTD-93-27. l\am II, 19W). Spedfte ««iqptoa of ih» pro|fMM dii£ shonkl noft 
have been ooukkred to be within dnadefhiii^ 

a faUom: 

« Rvtin Start Prof iam. White eeaployoeei tnOflint fliey be a Kcoodaiy 

EvasStKtproiram,iito««^>ctoitypufpoae. Raflur, the pwpoie of ftc profram 
ia family Ulnrvy anS ^K a ting ediratinti trainlfl|. 

9 TttlOPromma. nc puipoic oflUirwart Bennl, Taktt Sii«h. Sti^ 
Servkea, a«l BdoMlioMl Oppoitiaiity Ces^ 

youth ifld flunaiMiiilf coUeteliCDdesii lo coispkia •econdary aehool and bo eotoU in 
ptMlOTf iml My ^wfHff*^ protno4 by providiog academic nppon servioei aod 
lOfbcmariM oa fteandal aid. 

• Y^g^i iiA..^^ ^^^AAr^jjjgf^ Prf^^r^ TIm daft&itioa'f Usutitkn » 
^adaltt Mid oQt-of-«d»oot youiha* {eouhidM moat faidiTiduiili acmd by w 
Mhnifliatned pcosrama UAkr dM ^Idns Act wintaiw hi lecoiidanr ■chooh. Some of 
dM *"wt« awarded oader ttw vocttieoal ertnratioii pcograma lieiad ia t» appcDdket to 
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See comment 5. 



(be draft ttfioa coald be ustd tor adults or ouc-^f-school youth, but they were ckh 
appropmml for, lad coiiM noc he u$ed excliulvely for, ifaw populiriOBi. Thudisdng 
ibo tax icM^admiiutttfed voeatiodal educatiOG prosnms «^ ijacladiag tbeir 
ip pr o pc te fiooi la Appcodix 1 to Ifab draft report (p«CM 1^19) in ibe tottl for 
*6ix^pfaQiynMnt trtimjo^ assiiiince" )t veiy miOetdiot. 

• Cooocniive ivmrtiwrartftf^ PTflffTim Eveo ihou^ t&6 difinitioa tafen io piogrimt it^ 
provide "caplpymeia traimng,* tUe dlnft report tpecifica&y tmam tbe data coUectknx 
tfld moQitociDi actif itiet of tt» Cooperative PcoooQitratioa FioinBii of (tie PnrkiDi Act, 
which it not an caiployxiieat craiotaf pcofffisioa Tbta wia difatsaed with 7im 
OwcaizakofOAO'aDetiohlU^touaOffiDemate^^ 1, 
1994, witfi oMttfem of M Dflpdtmtttt. and Mr. Oii^^ 
by foQCnoie that coUectkm of ov4^n» data wm oot applleabto t9 piocram. A$ wu 
d i a cui a ed , die poipoae of ihii proknm ii to exlcDd nacoestfol leduii^ by having 
gramaai daooaatmi vdxtimr inrtinirinfi abca^r pnrrea to he foccenful covU be x»ed 
by odier eoAiea. "Omcodie dm'i teted to ipedfic wchokf^m to be demonatratKl 
is fathered both in die cpplkattODi for grant fuodla and in d» iodepettlent evahmtioa diat 
thttgranimareTeqQind &opcoivide. Sk 34 CFR ft 42tfai(b) and 42^.32. 

TIm r<mffitrinne m^ed In the dttft re|»ort ate vM ftccvate aad an band on Jaeempkte 



GoMcaUy, h ia difflcult for ua to delannio^ die ovwUl accuracy of the reporf a cooduaiaoa aa 
tfapy reteie to diia Dcpaxvnetit*a programs.] In aoma iacnooaa, wa casoot idcotiiy (be specific 
progiiiu liitDd m tcpon from die aaoiaa ^vco. Ia otter hMraiifi, twi do apt fliink that 
GAO kK>ked at aU die data coOeciDd by tta|s DcpenaiMi; and, readied inaoooraoe 
cmiffhiikini about tha outtoeae dahi bctog ^otlectKl in regaid to apedflc prognm. InadditioDto 
die iwh'iiiif and ieapiot eivahiiiioDi noted ^arliflr« the Depfftmeot coUecti TnTtp^?inr data on odier 
piugiauu. Specifip fxarrpiiia ef diaae pco^nina lochida: 

• S^03fid!^JBdDflBkMlL£*vttSL£EfitDB* AWwigh dw chart on pige 31 of die draft 
repoR ftadieaiM odiicwiae. die Wpmeo'a Educatoal Eqputy progitm doea pcovkle foor 
flawyiil nM'wilaa tag Md dia ootlertion of pattiuipam otftocsiip dati. The partic^paxst 
outmne data ooOeetsd k prhnarOjr data on tUna MUieimd , 

• Vrfv*^Hi^ ftfiiraKrtw ^Mr*M The conchiitott that the De y e ttuiad does apt collect 
cutDome data on vceadooat educajioa piograme doaa not reflect diat diat th£ 
>MqmI CnlK for Edacadon $t<iitka (NCBS). oot 

Edw igMP, le Midieih wdby aianitatQeoaacttoditeforti^ AidMfodaral 
leivL Md tpedffraOy for furpoeda of pnvidiDg iitfoiiiitloQtD Coagraae reievanT to 
pojbtTiwIdngt KCBS ia chatigad iMk the orcxaU leaposMdMUty for esiyiliildpg a tywssxx 
forooUkffdigpBiaQnVoeatkiQaiiBhicadQaiifldaraecd^ Ihis 
ayitawhei been in cwMnma siDol lSt7 and darirea data on vocateial cdararfion from a 
onmbinarton cif gmsl porpoaa edocarinn aorreys-die approach tmtkrtttA by tit Ftrldaf 
Act. 
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See comment 6. 



Tbt pnak9 of the dnifi rtport dots not reaw< th« sumxury tchttne of th/t Dcpartmest 
itate-wbnMiterMi pcogr«ni. 

Ovenll, tto dxaft repoci evidences a raiionbcrstiodlfig of the snunDry scfaemes tM. aaiborjzc 
f edjwil ftuxar^ ftir ia»-idmiait«red The 'btckground* sectwa of the draft 

report (pages V4) iwlic*tft$: 

"To mmtc prognuni fci the moct from thii inveatmect, piogiam admisiitraiDcs must 
hiveinformitionibouithtlrprotyiiM' Wilhthii 
ixignxuiicu, thdy can mdke chjiuttt to fe»prot« prograait, vodi modifying the 
tTpetorxmberioricrfkMSTlflibtotobelp p«tfciptntt leoeive irtiiiinc mat meM 
ibeirMedsattleQibleitbemtoobttiaettpby^^ [HmpfaaiU added.] 

GAO'i f^Tr«»^ t^M-mmf dw seed for Outeoi&e daia eoabk protntn admmittnton to 
-make chiapt to unproire procrami' evidpocet wine miiundexitaraizag of not oid^ the itattUovy 
MheEOMa of dM Da paftm a o t'i mit artmlnitHrrl prognms, but ako tlic linatid authorit y of d ie 
federal o£ffeUliiHio admhiltiac thasL la ceoertl, in ludurizittg ttate-adml^^ 
ate fiuiM on the bwia of MB plans, Coo^ has |Wes dia iudkori9 «> ttiks ftindh« 
daciakw with reipca to stale and, in lom* pioinuM local, ac^^ 

fMaral tram. As diacuiaad bckiw, a iMiffe extap!? of ttis ia the iti^ 
vocadondedlueaiteFrof^audiartzadb^ CooriMwidi ttiis, Consresshu 

reauM onnccM (tata xtlated to ti» partk|)^ 

o^lectodbydwmibgraiMDeaniUstibc^^ With ic^Kct lo die Mte-admiiiiilend 
vocadoml ednntkm profrxni. d^ Perkatf Act 

Thaiaiat-MtticDcpwlinent-d^ttnninafrf^ 

on d»ctquii«latalaa8KaanBata$al local cvabMt^ Under die Fteldni Am« d)i 
DepartoMa mu« diitribute funds fcr Ok stile^^ 

uate sBbmiis a Mate pfao duit mseb die roqulraiMts of eectioo 113 of dM ?ii^kini Mi 
aDdiiof9ufficicat<iiuaity. Sectte 113 isqtdras a ttats lo iMt ill w of ftndi o^ 
i«pxindataa»auNaiMot«nlfoaii. Tliw, due DapartBaat haa no au flwffl^ to 
other uaea of the fttnia or lo ledirkct f^jsaUng widdn tbe tiata to cdw actJnrldea. 

• SiaiBa<tevdopsiu)dacdsMKlnM*mwlkicfaicfl^ 

diBgo^attaaMdmMcoKdftKxWldiaeetkAllSordKltrk^ Itelocal 

i«dptain parftann ainial finhtfdm, 1^ 

paitk^Mtt. ttti devekp io^woveottii plam If aufattan^ 

luctiiv d» itaMi itBttUfda aal n^aauret parittant to lec^ 

Dm Act doea wt reiyufe dut scataa fiport oulccae dim to the Depanacix. 
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DEPARTMENT OF HEALTH ^ HUMAN SERVICES 



Offict of ln«p«ctor Q«n«ftl 



WMhinQton, D.C. 20201 



MAR 2 1994 



Ms . Linda G. Morra 
Director, Education 

and Employment Issues 
United States General 

Accounting Office 
Washington, D.C. 20548 

Dear Ms . Morra: 

Enclosed are the Department's comments on your draft report, 
"Multiple Employment Training Programs: Most Federal Agencies 
Do Not Know If Their Programs Are' Working Effectively." The 
comments represent the tentative position of the Department and 
are subject to reevaluation when the final version of this report 
is received. 

The Department appreciates the opportunity to comment on this 
draft report before its publication. 



Sincerely yours. 




Enclosure 
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roMMfHTS OP THU DEPARTMENT OP HEALTH AND HUMAN^^SERVICES. ON 
Tj-HE U.S. GFWERAL ACCOTTNTTNG OFFipE'S DRAFT REPORT. ••HULTIPLE 
T?MPT^YMENT TRAINING PROGRAMS: Mo«i: Federal Aa«ncl«ft DO N^tL KHQW. , 
If Their proorams Ar« Workina Ef f actively* " 
Report NO, GAO/HEHS-94-88 




Thank you for the opportunity to review your draft report. 




The Job opportunities and Basic Skills Training (JOBS) Program is 
correctly shown in table format, on page 11 and page 32, as 
having the program monitoring and evaluation found absent in most 
other listed programs. 




General Comments 


See comment 7. 


We are concerned that the report's definition of ••employment 
training programs" is too broad to be useful. Many of the 
programs listed do not have employment as a specific goal. The 
narrative is brief and sweeping in its criticism of Federal 
programs without acknowledging that employment training is not 
the primary activity of many of these programs. We suggest that 
the General Accounting Office (GAO) try to categorize the 
programs and concentrate on programs with specific employment 
training goals in this report. 


See comment 8. 


In the executive summary on page 2 of the report, GAO states that 
they did not attempt to determine why agencies did not collect 
data on participant outcomes. It would be helpful to have at 
least an initial determination of which programs are required by 
statute or regulation to undertake such activities and to what 
degree these activities are required. 


See comment 9. 


We disagree that the lack of data on participant outcomes 
automatically means ••Most Federal Agencies Do Not Know If Their 
Programs Are Working Ef fectively . •• There are numerous completed 
and ongoing studies that have evaluated program effectiveness and 
have provided sufficient feedback to give some indication of 
program effectiveness. As part of the report, GAO reviewed many 
of these studies, some of which are summarized in Appendix V of 
the report. We are concerned that the title of this report is 
not an accurate reflection of its contents. A better title would 
be "Most Federal Agencies Do Not Collect Data On Participant 
Achievements. " 


See comment 10. 


State Legalization Impact Assistance Grants (SLIAG) is listed in 
Appendix I of the report as one of the Federal programs in the 
Department of Health and Human Services which provide employment 
and training. It also includes the full amount of Fiscal Year 
1994 grants to States, suggesting the full grant amount is used 
for this purpose. We believe that the draft report is not 
correct on both counts, 

1 

1 
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See comment 1 1 . 



SLIAG Is not in any way an snployxnent or a training program and 
should be deleted from the listing. Even if the progreui is 
included^ it is misleading to list the full amount of grants. 
The program provides funds for public assistance, public health 
assistance, educational services, employment discrimination 
education and outreach, Phase 11 outreach# SLIAG administrative 
costs and program administrative costs. With regard to adult 
education, the regulations for SLIAG at 45 C.F.R. 402.2 allow 
reimbursement only for adult educational siervices authorized by 
the Adult Education Act (P. L. 89-*750) as in effect November 6, 
1986. Guidance provided by the SLIAG program to States on 
October 21, 1988, indicated that vocational education services 
were not authorized by the Adult Education Act, and therefore 
cannot )m paid for with SLIAG funds. 

Three Refugee and Entrant Assistance programs are also listed: 
Discretionary Grants, State. Administered Programs, and Voluntary 
Agency Programs. These programs offer a vide range of assistance 
and services including, for example, direct income maintenance, 
services for the aged, and medical translators. Job placement is 
a major objective of these programs, but y>Qne of these programs 
is primarily an e mployiBent training proyram. 
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U 5 DFlP^frTWENT OF l-ABCR 

WASHIKaTON. O.C 



February 24, 1^94 



See comment 12. 



Ms Linda G. Morxa 
Director 

Bducation and Eoploynervt issues 
HuDian Reaourcea Division 
n.g. G«naral Acccointing Office 
4411 Q Straatf K.X. 
Waahington, D.C 20548 

Dear M«. Korra: 

^! fflIS !^ |l?Cinff g?f."i^v. we your proviaing 

tba opportunity for ua to conmant:. 

aarvicaa . 



Siyncercly, 
Robert B. R«lch 
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0,3, D«p«rt»«nt of Labor 
vumgonmm to Draft GAO a«port 

Host; ratenl Agenci** Do 
mmXtA CIO to d*t«Xttintts 

o IrtMit data f«d«r«l miQDci9» collect on pwrtlclpwit outooiiM. 
o Hcnr f«d«»l •omi^m wemitor loo*l pitigrw p«forwi»o#. 
o lOiat studiM of progw» •£foctlv«MMi» hxwm oowtnotwl- 

Tiftriftr*'' riMi 

«rt>A n^BAi^aattti aoTMa irlth GAO on thm iBportano* of collecting 
2S ;^f infommtion. th* f^^^tX^^JL^lo^ 

Staa&^obl«sA ilthln the cont^t of a rlgorow isnO^ and to 
fUnA vbat iforks. 

warm. tbA tood9«t »l«o «i9pcnrt« a propoaad nay * ■ iSl^S 
anLatoDOArai^ omSiucv, and an aipaaaion of tha Job 
^rataSf "•^J^-T^^r^ «MrlM to stzavMlina today 

pitflbMorlt of tralalag progrM— «« «ot«! in ttoia mX jaittoa gao 

tttpoxta* 

na bavm baon »aarr*i1iw ftor a vaxiaty of evidaaoa ot jfimt wrfca 
SiSSS'ln aarly Mmwqr im 52^^ /,Sffi"!SoS 

AMTioaVi tbat bioag^ waaagara and «w<»?«?^2£,?2KJ 
^^naa tSatbarTlCba oo3aranca not ottly hiflhligbtad prognM 
S^Srlt toutfocuaad on ld«tifyin« tbair critical oo^m 
ali 



wm 1 iMir tMBoaal alao addxaaaaa our aooountability for 
25l2-i^^ffi*^«^*SoS^ diaadfantagad--«dttlta M 

M ^T^Tt^^i^ tbat ia tha focua of tbi- 

avidMwa and o«waxtfe«ita raaonroaa vbara tb^ * fS^S 

S inStrata, a aajor r^^^^J^.^JL^l^^^ 
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See comment 13. 



See comment 14. 



IX tr«ioiQ9 prograM tox tf^t-^t-Mdiool aisaitv»nt«9ttd youth fail 
to povitiv* iji^pmft on oaploywit and Mumlngs. Our propoMd 

budgvt outfl iqpmdixig ea tteM «ppro«GlMS-»«irhiX« ve d«v«Iop immt, 
von ^ffoctivtt iMPproftcbM^-vliila inorouliig tlMi badQ«t for adult 
ptogr m MS fer tte dijMUltv»iitik9«d «nd for Job Corp«« mxmtim vtMm 
awAluatioa tas nhovn p ro g xa — to )>• oo«t-<i>«iHi£iol«I« 

ffa «l«o •gzw vitti GAO that oollaetloa of Inforaation on outcoMa 
«nd mffotivwommB im jUnportant. Within tha ponstraints of th« 
diaftiniflhod ro«aviro«« that csongr«M hMm K»da aY^llabla for this ^ 
purpoM in racMmt ywrs, imi tuiv« •oug:ut to oollaot and vmm. thu 
tmmt iJtforMtioa can* Ha uc« pXttaaad, ho«P«var, 1^ ooo 
^•aront turMrouad in support tor this motlvity. 0«r FT 1994 
appropriation has providad additional trraluation funds that 
p««mitt4id us to lauaeh a nair random-assi^aasnt «f faoti'vwDass 
stvalnation of Job Corps. Vs ars bopsfuX that a siaiilar r sq o a st 
ror n 19SS viu also bs tnactad, anabXing us to oont i iwi s ths 
study, nas GM study «ay support ths aXlooation of rssonross 
that ar« o ssds d to rcsisdy ths currsnt widar^iixvsst»ant in ootoons 
and sffsctivsQsss inforsation* 



oao fo ous ssd on «3 progrsss — out of a uniirsrss of 194 asplovKsnt 
training prograns tiMV bad prsviously idsntif isd— that provids 
*sc«s" ssployMRt training assistancs to soonoidoally 
disttdvantftgsd parssns. tbat subsst oootainsd it DOL prograais, 
out of 36 DO£i prograas ywsifslnsd on ths longsr list. 

CsMsntt Of ths IS prograns^ im do noit oonsidar f Ivs (5) as 
appropriats to ths rsport*s focus oo sai^loyMnt trsioing 
asslstaaoa to ths soonoHiflally disaAvantstfsd. Xhras substantial 
proqam that GJU> inoladsd«»<Vrsds Idjustasnt hssistanos and t«o 
typis of ysdsral allotMats to siQport stats laployasnt Ssrrios 
• aotiTitiss-~f inanos ssrrioas and sotivitiss that havs no inooM«^ 



sllgibility roqolrsMnts for rsolpisnts of ssnrloss and azs not 
dssignsd to focns on soonosloally dissdrantMsd parsons. To ths 
oontrary thsy assist serlosrs froca a vids varisty of s oc p oni o 
cirousstanoss who bavs lost jobs and/or ars sswitig to find 
bsttsr lobs. 



TWO otbsr *pragrssfi««-~7iiot and Dsnonstrmtion and ons of its 
BMll ooivonaots (tbs ysdsral Bonding Prograal— ars aot dsslgnsd 
to prorids sssistanos but rathsr to pilot, dsnonstrats and tsst a 
irarin^ of Innowativs progrnn approa«iiss« Shsy ars not rsally 
yrogi>m^ In ths ssy that vs undsrstand ths tsm is bsing ussd for 
this rsport* IN Liiii.—and that CM clarify its dsf inition for 
progran inclusion in this rmort snd tate a dossr look at 
shsthsr tb sss fiYs pregrans bslongx us bslianrs that thsy do not. 

Ons progran that should bs AOdad to ths list is ths Bonslsis 
Vatsrans Xsintsgration Projsot. it is listsd in Appsndix 1, 
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See comment 15. 



Now on p. 8. 



Deleted 



See comment 16. 



servM eoonoaicaaiy disadvantaged pmsmanm, and i» a ••P«»t« 
program tv<m SOA'a Job Training for the KoMl^taa Dcaaonatration 
PFograx* 

Collacrtio n r^^nfw^ n«ta and Cnndiict of fttwIltfW Qff PTOflTMi 

Wffagtlvnar* 

GAO p o n rfl mltff that ••moat aganciea db not ooll*et in toria atlon on 
prograa outocwMi nor do thay oondoot studia* of program 
•ftactivmaaa." 

OOMMts Vftvttthar or not thia atatanant !• txna o£ moat aganoiM^ 
wa do not baliava it ia trua for tha ta)>or Oap^tMmt. »i 
bali«va tbat tUo DapartMnt deaa « ^food jed in this zaapwt. Of 
^ 14 Daparta^ piognov that az« fooaaaad on tba acoiK«ioally 
diMadvantagad. tba Dapurtaant ool.lacta ao*a pzognnr-oatoaMa 
SoSSioToi 11 orS« «a indioatad 

ttaa Diaablad Vataranm' OuctraaOh Vrograa B aaa l a aa^ itagana 
xaintogratlon Projaot, for a total ofi3. |ia ham . 
vtndiM of prograa affaotivanaaa on »o«t« 1»a Xiat atodlaa that 
caootaia aff aoUmnaaa laforaation an thaaa pjt i jgi.aM in an 
attaohnant* 

Thar* ara two nlaooaoaptiona that l^ad GAO to «»5S^****.Sf?*v!^ 
asoont of outocsa inforwatioo and af faotivaoaaa atodiaa that hava 
baao dona on prograw. 

on aataoM iafaxHUaa, aa tha raport 4»^^uotoradj aotj^ 
aiatakaalY link oollaotion of outooM laf^wtlon monitoring 
aetivitiaa. In faot. moat o^^axtmaot Vvograaa oollaot 
partloipant'-ootaoM infozmatioo thzongh raportin g ayata ma> 
Monitoriag oamot naually ha imivarmai, ?oo« rapoctlng ^ 
nauaXiy can. Wm ara not surpdaad that ^e ahla X 
momt programs thronghoot tha Vadaral uuvaLiaamt 
partlfiipant-oiitooM data thvo««b monitoriagr that 



{p\ ») ahoma tbat 
do mot obtain 

„ „ _ _ ^ ia mot a vary 

^ioiaSt may of Obtaining oomplata informatian. iha aaaXyaia 
vxixaa up tha maana ofravlair. lafrliMr af ontoomaa ia typioalXy 
tLroogh^atatiatioal analyaia of rapoxting aystaM^ not om-aita 
anaXyaim that ia uaoally mora foouaad on ooaplianoa iaauaa. 

A claar axaiv^a of thia OM^a oonfoaion ia on p. 9 of tha draft 
raport abarm CftO diaonaaam tha monitoring gnida for tha Job 
nainiag for tha moMlaaa oaaonatration Program and notaa that it 
doaa i9ot oovmr partiolpant ootooman. 

In fact, quartarly ontooM data from all gxantaaa ara ragairad toy 
tha Dapartmant tialiw a raporting fozm that ia aivportad by a 
oMprahanaiva oliant-laval information syatM. k copy of tha 
form ia attached. Tha Dapartaant doaa tooir vhathar partioipanta 
find j^a— oontrary to tha atatamant on p. 9* Xndaad Appandlx xi 
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See comment 1 7. 



See comment 18. 



oorrfwtly nPteo (on p 29) that «.ploy«nt-.t«t»i« intorMtlon I. 
•vallabl* for tM« pregru. 

.ffiiriTMW stirfiM. <aO foUom * v«ry narrow dafinltion of 

^ooJc^mm^ot-^tm « partlcipwit group and control «ro«p. 
ut.'ii* DMiartMnt hu piooMXvd in tba of •nob »tttdi«*— 

SSrinitr.SUiM of Di5«rtMitt«i proa**-' 

i^iotiTttt^s of Bany typaa of pxogra^i {a.?-, raaaaxot and 
dawMMrtration progzaaa]. 

oonaidarabla l«fot»atlot» on •"•«ti^««Ji!_2S-^i!l^S2 aftar 

SaiiSti^TSlyala. •2« "?^J''t^2*:LSS^^ 
rbm DapaztMat baa eanOuctad a rarlaty of auch "ty ^y 
™iSraf«aotiTanaaa iatoaati oa that S«X|5 '"f^L?*-- 

•ceaoBlcally diaadwantagad la listad In m attacdaMot. 

ilata daaoriptioa of 



in ow oplaiaa, (UO ahovld pwwldaa «SiJ|* 
dlf faraS^ typaa o£ affactivaaaaa ■^■^••'-^i^?;}^,*** 
■traagtha aadifaakBaaaaa—lsafora dzmring oanslvaloMi- 

Tlta All 



& final t«Mxal 



is • 



aisa fits aU" or "baaa oouatlng" 



dltfarafwaa ia SSr «l4*etlvaa and stwotuw— laoiadtoj «ja 
natacaofLsagaMtit and raportlng ralatiagafclps iMto aaa tta 
MaavZi iM^liStbK Bzo^M OVsntors. «e iUnstvata, in Job 
22S?ttSlSSl5lS^o^tSi^thitpro»i«a ssrvloas Miya dlc«:t 
f^^SaSS a^SrManieo^«»antI abacaaa OTfk Titla ZX is 

aiiiau MMl /imrrl T<l»t xoia. 

na tasMfs andifCsxantiatad losing togatft^f^prOTSswltth 
vary dLUarwt purpoaas, dltfarant foras of Fadalfal iwrolraaant 

siss diffSantlaU doaa not »-iPj?« '-^J^.-g: 
■aanlnoful diatinotioiw and randara any ococluaiona at tba 
aggragats l«»al of vary ll«itad naaful— 
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Now on pp. 6-7. 
See comment 19. 



See comment 20. 



See comment 21. 
See comment 22. 

See comment 23. 



In 



Pii 7-.t. GAO suMMtfl th« desirability of linking data on 
SkLiiantctMSSo^ training provldad, •^Jf'^^^r, 

^S^^SS^f^^mm^iM a B««aa5»a«it.inXor«i«tion .y»t^ 
SSt iTSSSS. of til* «^«>*«-ttf>ulatian« cugga^t^ 

^ JSSSTor bstni «ct«dpa to our JTWi Indian and »*tiv« 
Aaarican Program. 

^,^m^ virnrM^rmr that vooh a »y«t«i c«n b« MtabXiflbod for 
SiSiaSto typl«mlly allooat* to mcb «ctivity. Unit** 

5SrS^ •ftort. CJ«> ftiU to oea*i4ir th« i»«t 

dlMiMloiui Of th»lr woric. 

thM UP tia« imy— «or woaipl*, listing OTOA IIX ■^iiiL^itM? im 
SuMScTSogri*.. 1» on. St«t« .•t-Mi<to progroi that 1» 

inBia«nt«lly tund*d ttam two aoozoM. 

P ai. "itoidlJia flgvtnw for Tltl« IH for -1994- ar. 

DUoiltlaBKy-. The^ figure *> not ocmpmtm tQ OJI- " 

iliailaS, iiMt N» •lloo«t«d to «ib«t»t« grantee*. ritXm III w» 
■ub*tst« BTMs. not ««r»lo« D«X1t«T atmb. 

P 22. Tbe "B^wtMy's Dltcrrtloiun^^ 21gur« ($114.7 ■illloa) 

tSijab !»S3«ig for tho «*•!••• DMor«tr«tloo Frogrw ahoia* bo 
$7.5 ailtlon. ThMo |u:ogj.*— or* «l«tl«ct. 

'fi^-^TZpSSrtStSi'S^xJ^ ii^L-SS^rt;- s-s.%r 

iTShlSTtSS^ 5p«»Pri«t«l- miw i» no .pproprUtion for 
ciMB Air iB rr km. 

XX. Hhy doM thi« .ppmdix not Il.t both OTPA TiU« IIC 
•ltd Titl* IIA, mm AppMdlx I do««? »ood «»Mii«t«»cy. 

p. at. InULeatM that th« D«pM:t*i«t of Utoor «»oo«jwt colloct 
iifM«.Son^S^I«rtioii«nt SSll .tt«li»«»t- f or JTPX llA 
l^UStlwgr«nt ScogrMST Thi« Im ln«»rr«et» all o»t«i^»for 
lSS5wu«ir?^loip«tlng in project. fu«S«l by lno««tlv. gr«t. - 
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See comment 24. 



See comment 25. 



- iDcluding mklll «tt«iRwmt«'^r« includes in tb« JTPA 
«dmiAijitxmtiT# rttporting mymtm- 

V* 29, Dlwa»l«4 V«t«wui* Outr«acfa ProycSi. XndloHtMi that VETS 
do Bfit routiiMly o^U«ot daU on MployiMnt status, slOll 
attaimHEtt, «M vage lamia. Sba VITS raporting ayataa la pairt 
of tba bployaant Sarvloa raportlxig myiftm and tba CSMO raport 
cx«dit tha n Stata granta prograa aa including aat^loyaant atataa 
data. Wa baXiava, tbaraforar that aaployaant atatoa abould ba 
incladad for OVXP. 

tha Wcaial^B Vatarana SaintagraUon Frojaot ahotQd ba liatad 
banaath tba armxP and tba ooXwna "aiiployaant atatna" and 
opartdoipant vaga lavala" aboald ba cbacload vitb an X« tula 
infoxaation la oollaotad on paxtloipinta and bad baan ainoa tha 
baglanlng of tba pxrogm* 

appaftdix XIX 

V* aa. Indicataa that tha Dapartaant of liabor doaa not aoaitor 
paztloipant outooHaa for STPk Titlm Xth diaadvantagad adult 
prograaa and for axsh ilk Inoaatlva-graat prograaa* nis la 
liM^niKztf participant oatooMa both for diaadvantaaad adalta 
and iadividnala partioipating in inoantlva grant pvojaota ara 
xapoctad annaaliT in tha %-^w«fi.» aystaM« 



Tba OtpartMot, tbxooQh ita hagional offioaa, alao oonduota 
oageiag raviawa and analyMa of prograa otttcoMa by itata and 
Xooal Sim^ InoXwUng progru achiavaaanta against part oraaaoa 
atandarda. taMxy raporta ara providad to Oongraaa and ara alao 
inoorpormtad into tlM Chiaf rinaaolal Offieair*a raviav prooaaa* 

Aa a rMilt of tha 1912 JTPA Bafora toandatnta, atataa ara now 
ragairad to Mtait raporta to tha MtionaX off ioa on indlTidual 
aa (SU) parfara>mna highlighting vhioh SDM Mb or did iMt 
thair ataadarda— AM daaoriptiva InforMtian aitplainliig abat 
tachaloal aseiatanoa viU ba providad to undar-parfoniing local 



P« IS* Participant O at oc— a for tha Job Training for tha 
■faalaaa nanaaiialiun Program ara aonitorad throoga ita 
goartavlv «ayort1»g ayataa, aa diaouaaad aarliar* Add an X in 
tha pamolpast otcoaaa box. Thara ia a apaoifio h ooi t oring 
guida tmc tha ■oBalaaa Vatarana Kaiatagration Projact that tahaa 
into anocif oaifaiaooe atatua and flnanolal nonitoring. Thia 
" ' ba liatad aaparataly» 



p. 43. Iba atndy liatad nndar Job Oorpa rafara to tha JOBS 
progran. It la not a Job Oozpa atudy. 
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See comment 26. 



Now on p. 44. 
See comment 27. 



See comment 28. 



Mint«gratioa Project ttudy« pr«v»a« W T a n a n l — 1 A —iiitanoa yd 
Trminlng Oozpox«ti«n vater •ovtiMt vitt WlOK/IPMi. mis «tndy 
vafl txaaaalttsd to €tmfrrm mm m^mmitUA in 1ft* W i ^lw wi y Aot* ^ 
cm U av*ilabl« on gfu— ^. tKmm mUkmm B u — n« at 219HIU0. 
8h« M7 b« contaotM if thmxm mxm mmr ^i— ti o n rfrtHwtf tb« 

BVRP. 

lypaidiK Y 

P. 50. Sha mot4kl findiAf tbat HaHla y tcmiaiiiff BMidatojy did 
not hftYs a aignif leant lJ«MMit aa tte a«tiM%ad aM^loymant and 
aaml»ga diff«raiicaa of rVL. tw i i jiaaa and Via jpglyiawfca^^' la 
aoourata ooly la a aarrw aanaa, it ia tia i ii an* aC aaatart and 
la ttlalaadlag. 

Tba laportant tlading, mtit MaatiaMd^ la ttat ^ •tojf cidK ilntf ad 
ttot tKa txalalM vacfolraMats i) raiaoad aw a it af r«:al9tt 
2) lad to a dooXlaa Li tha laaftk at Initial ^paU of 
jatoleaaiMaa asid 3) lad to aa i n a r a i aa Ia aa nia ^ i ^ 

Paga lb* ■iwiijj la of aa aaftlnar.dsUMrMt atady froat 
■oaann*a. Tfaa xafarano 

Attaotaanta 

jnv CaaxtacrXT Mipartlnf 
Mat of af factlTanaaa raXatad 

/ 
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APPENOJX 8: JTHDP QUARTEW-Y RCPORTINa fCMW 



Job TrMiQ for tfwHonMiMs 




k »tiiMU i»M3cy 



I mwte iMuMxy owi^f 







a-t 



PFST COPY AVAILABLE 



Page 77 



GAOAlEHS-94-88 Multiple Employment Training Progranw 



Appendix VI 

Comment From the Agencies 



Vourt««n (14) Labor D«partMnt Ci^oyMnt: Training 
For tha loonoaloally Dxaadvantagad 

TMpiTOTlM tha QQa3i fcy of yra<«in7 ftrxi^T' jtoa. BartoXvy Plaining 
Asflooi&tas and WKl intamatlonal, 1S99. 

Praefcleal oniAmnr^ rr^y af»ir«inflt^*" W TiKitMii^rY Councilfl. 

Ont, Zso. 1990 • 

Bartelorf PXatmixm laaociatas, 

gwftifyirm^ WOmttflf 'Kbt AvMciatas, 1993. 



Sitlft_II=&« Abt teaooiataa, 1994. 



Ganaral JU30oimtiA9 Qttltsm, 1997. 

v^i, frwriM, U.S CtKtyal Acoo«utlM| Offioa^ 19«9« 



rf tiM iTTtli Titlft 7^ Program, Barkalay Planning 

Htwoniat^ X99»> 



JTPA Joil 

Mathaaatloa Pollcay RaMarcta, 1982 « 

Yflwtrh ftmlttvmanfc mtwt Training Pfotfr^ai CTPya Ta«t^, riational 

RaMarch Counall^ 1995. 
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^^r^ tH^f dy of th« rt pliiMr»*»^»» TMttwfd Job* Tax 



Pelicy KaRTf^w «Hd avatl»«irttm of tAa Ttowt^ Jdbfl Tax Credit. TvT 
AsaoolatMi, 1992. 



XsiCf m.o.V 8oi«noaa, 1991. 

T>^tni«ef for «ia gr«iiiMi nnfinitnl-lfTn rrrtip-iM ^aaas b«ii 

A— ociat—r 1993. 



wa^i^M v^^tt> fa<*>f9grBt1nn gfamv. vaolmical Mslataaoa 

and Training Corporation, x»92. 
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UNITBD STATM 
OFFICE OF PERSONNEL MAHAOEMENT 

WASHINOTON. O.C. a041ft 



orFiCK OF TMK DIKKCTOK 



EB I 7 (99/1 



Me, Linda G, Morra 
Diractorr Mucation and 

Enployaant Issuas 
Human Rasourcas Division 
U.S. Canaral Accounting offica 
441 G Straat, NW. , suita 650 
Washington, DC 20548 

Daar Ks. Morra: 

Thank you for tha opportunity to raviaw tha racant Ganaral 
Accounting Offica raport, "Multipla Eaploy«ant Training 
Programs. « 

Wa hava read tha raport and hava no comaants. 

Sine 
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U.S.O«partm*ntof 
Trontportatlon 



Ass'S'af^' Seceia'y 
lor Ad'TitniSlfaiiO'" 



AOC Sftvtnih Si S W 
WHSMngion DC 20*i90 



February 25, 1994 



Ms. Linda G. Morra 

Director, Education and Employment Issues 
U.S. General Accounting Office 
441 G Street N.W. 
Washington, D.C. 20548 

Dear Ms. Morra: 

The Department of Transportation offers the following comment regarding the 
General Accounting Office (GAO) draft report titled "Multiple Employment Training 
Programs: Most Federal Agencies Do Not Know If Their Programs Are Working 
Effectiveiy." HEHS-94-88. 

Appendix I, page 23 of the draft report shows one program for the Department of 
Transportation identified as "Human Resource Programs." with planned funding for 
Fiscal Year (FY) 1994 of $^5 million. We understand from discussions with your 
staff that this citation- Is intended to represent training programs for the 
disadvantaged conducted by the Federal Transit Administration (FTA) under 
Section 20 of the Federal Transit Act as amended (49 U.S.C. app. 1616). We have 
concluded that the $1.5 million funding shown In the char: is actually the FY 1993 
funding level for all human resource programs conducted i^y FTA. Only a portion of 
these funds were devoted to job training for the economically disadvantaged. For 
FY 1994, the proposed funding level is $700,000 for the FTAs human resources • 
program,' with about $600,000 directed at job training for the economically 
disadvantaged. 

Thank you for the opportunity to review this report. If you have any questions 
concerning our reply, please contact Martin Gartel on 202-366-6145. 



Sincerely, 




Page 81 



S 2 GAO/HEHS-94-88 Multiple Employment Training Progrania 



Appendix VI 

Commenta From the Agencies 



GAO ConniTlOntS following are gag's comments on the letters received from the 

Department of Defense, Department of Education, Department of Health 
and Human Services, Department of Labor, the Office of Personnel 
Management, and the Department of Transportation. 



Department of Defense l. The Department of Defense disagreed v^dth the inclusion of the Military 

Base Reuse Studies and Community Planning Assistance Program in our 
analysis. This program is designed to help state and local goverrunents 
plan for the economic reuse of closing military installations so as to best 
suit the community's need for economic development and jobs. We believe 
this program also fits our criteria in that its goal is to enhance economic 
development and employment opportunities. To clarify v^hat types of 
programs v^^ere included in the scope of our v^ork, v^^e have modified the 
description of our criteria. 



Department of Education 2. The Department of Education expressed concern that it had not been 

given the opportunity to comment on our previous reports concerning 
multiple employment and training programs. For the testimony given June 
18, 1993, and the reports issued January 28, 1994, the requesters i^ 
specifically asked that v^^e not obtain comments firom affected agencies. 
Because this report identifies vi^hich programs collected participant 
outcome data or conducted effectiveness studies, the requester agreed to 
provide affected agencies the opportunity to comment. 

3. The Department of Education expressed concern that the report does 
not reflect the Department's recent initiatives to address deficiencies in 
the collection of outcome data. We recognize that the Department has 
several ongoing initiatives. However, because these initiatives were in the 
early planning phases, we had no basis for determining the extent to which 
these initiatives would overcome deficiencies in the collection of program 
outcome data As for the six evaluations identified by the Department, the 
study of the Even Start Program was already included in our listing of 
studies shown in appendix IV. We have modified the title so that it can be 
more easily identified. We also added the National Assessment of 
Vocational Education to our listing, although we determined that the study 
does not use a comparative analysis approach and could not be 
categorized as looking at program effectiveness. The other four identified 
studies are still under way and could not be analyzed for inclusion in our 
listing; however, we have added them to our footnote of ongoing studies. 
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4. The Department of Education objected to the incliision of six programs 
in our analysis — Even Start, Upward Bound, Talent Search, Student 
Support Services, Education Opportunity Centers, Vocational Education 
State-Administered, and Cooperative Demonstration. We included these 
programs in our analysis because they meet our criteria of providing 
assistance that enhances skills or employment opportunities. We have 
clarified the criteria used in determining which programs would be 
included in our analysis. We recognize that the primary purpose of these 
programs may not have been to provide employment training assistance; 
however, the assistance provided does help participants improve skills 
that meet our criteria. 

5. The Department also raised questions concerning specific data shown in 
appendixes II and III regarding the Women's Educational Equity Program. 
We have revised appendixes II and HI to show this program does collect 
participant outcome data on skill attainment and does monitor financial 
activities. The Department of Education also expressed concern that the 
report does not reflect data collected by the National Center for 
Educational Statistics concerning Vocational Education Programs. While 
the National Center for Educational Statistics does gather data on all 
vocational education programs, through a national sample, these data are 
only useful on a national level and cannot provide program administrators 
information on what happened to participants in a specific program. We 
agree that this information is a valuable source of data for the Congress 
and other policymakers, but it does not provide the specific data needed to 
track participant outcomes from a specific program. We have footnoted 
appendix II to indicate that our analysis does not include data collected by 
the National Center for Educational Statistics. 

6. The Department of Education expressed concern that the report does 
not reflect that some programs are state administered. We recognize that 
some programs are state administered and have modified language in the 
report to show that for some programs, federal program administrators 
can only make suggestions to improve programs, but they cannot make 
specific changes at the local level. However, we believe the Department 
has overall responsibility for managing its programs and assuring that 
programs get the most from the federal funds invested. 
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Department of Health and 7. The Department of Health and Human Services expressed concern that 
Human Services criteria for programs included in the analysis were too broad and 

included many programs that do not have employment as a specific goal. 
The criteria we used were quite broad because state and local projects 
often use resources from a wide range of programs to provide assistance 
to participants, particularly to enhance basic skills, including literacy and 
math. To fully tmderstand the breadth of the problems involved in 
coordinating such activities, we have included all the programs that 
provide such assistance to adults and out-of-school youth, 

8. The Department noted that we did not attempt to determine why 
agencies did not collect data on participant outcomes. We agree that it 
would have been good to have obtained information from agencies on why 
they did not obtain participant outcome data; however, because of the 
number of agencies and offices involved in the administration of the 
programs in our analysis, we were not able to obtain this information. 

9. The Department disagreed with our conclusion that most federal 
agencies do not know if their programs are working effectively. They point 
to the large number of studies listed in our report as evidence that 
agencies are getting some feedback on their programs. The conclusion of 
our report is not based solely on the lack of participant outcome data, but 
also on the lack of effectiveness studies that compare participant 
outcomes with outcomes of similar nonparticipants. Few of the studies 
listed in appendix IV were effectiveness studies by our definition. 

10. The Department of Health and Human Services questioned the 
inclusion of the State Legalization Impact Assistance Grants in our 
analysis. We agree that the primary purpose of the State Legalization 
Impact Assistance Grants is not to proviue employment training; however, 
the program does provide assistance that could enhance participants* 
basic skills and make them more employable, which meets our criteria for 
inclusion in our analysis. We recognize that not all the funds proposed for 
this program will go to provide basic skills, and, when possible, we have 
a(^justed the level of funding to show only the amount that would be spent 
for these types, of services. In this case, we were unable to obtain 
information on how much of the proposed funding would be used for this 
purpose. We have expanded the footnote to appendix I to reflect that 
while some amounts shown have been adjusted to show only the portion 
of the proposed budget that goes to serving adults and out-of-school 
youth, other programs are shown in full, even though only a portion of the 
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program funding may go to providing employment training assistance as 
defined in the report. 

11. The Department also questioned the inclusion of three Refugee and 
Entrant Assistance programs. The Department recognizes that job 
placement is a mgjor objective of these programs, but they state that it is 
not the primary purpose of the program. As stated previously, we have 
included those programs in our analysis that provide assistance to 
participants that enhance their employability or employment opportunities 
regardless of whether it is the primary purpose of the program. These 
programs meet these criteria. 



Department of Labor 12. The Department of Labor expressed disappointment in not being 

provided the opportunity to comment on two earlier reports concerning 
multiple employment training programs. As previously stated, the 
requesters for these reports specifically asked that we not obtain 
comments from affected agencies. We welcome any comments the 
Department may have concerning these reports. 

13, The Department of Labor questioned the inclusion of five programs in 
our analysis — ^taa, two fundr^g streams that support the ES, and two pilot 
and demonstration progrojns. They expressed concern that the taa 
program and the es funding streams are not designed to focus on the 
economically disadvantaged and do not have income-eligibility 
requirements. Our analysis includes any program that provides 
employment training assistance to the economically disadvantaged. We 
recognize that these programs may not focus specifically on the 
economically disadvantaged; however, many economically disadvantaged 
people receive such assistance from these programs. Therefore, we 
believe these programs should be included in our analysis. The 
Department states that the two other programs are not designed to 
provide assistance, but, rather, to pilot and test innovative approaches. We 
believe that while in the process of piloting and testing approaches, these 
programs also provide participants assistance that meet our criteria and, 
therefore, we have included them in our analysis. 

14. The Department of Labor identified the Homeless Veterans 
Reintegration Project as a program that should have been included in our 
analysis. We agree that this program appears to serve the economically 
disadvantaged and should have been included in owe analysis. In compiling 
information on all the programs funded by the federal government that 
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provide employment training assistance to the economically 
disadvantaged, we inadvertently omitted this program. We have noted its 
omission in appendix I. 

15. The Department of Labor suggests that we may have understated the 
amount of outcome information collected by agencies because it 
mistakenly linked collection of data with monitoring activities. Our 
analysis of agency collection of outcome data is based on two approaches: 
(1) outcome information gathered through reporting systems as discussed 
on page 5 and shown in Ifigure 1, and (2) outcome information obtained 
during on-site monitoring visits. We agree that the collection of outcome 
data is more often accomplished through reporting systems, but our 
analysis shows that only about half of the programs collect outcome data 
through reporting systems, which the Department described. 

16. The Department of Labor noted that our reference to the monitoring 
guide for the Homeless Demonstration Program did not take into 
consideration the quarterly outcome data reported by all grantees of the 
program. As shown in appendix 11, we recognize that the Homeless 
Demonstration Program does collect outcome data. We have deleted the 
reference to this program in our example. 

17. The Department of Labor expressed concern that we used a narrow 
definition of effectiveness studies. We agree that the definition used in oui* 
analysis is strict. However, random assignment is the only statistically 
valid method for truly knovraig whether a treatment is effective. We agree 
it is expensive, dijBTicult to execute, and does deny potentially beneficial 
treatment. We have added language to the report to reflect these concerns. 
The Department also listed several "effectiveness-related" studies, which 
they identified as being useful to their program managers in learning how 
their programs are working. We fourid that many of the studies listed were 
included in our listing as well. Tne otiier studies listed probably were very 
usefiil to program managers as were many of the other studies identified in 
our analysis. However, these studies do not appear to meet our criteria of 
an effectiveness study. We are not suggesting that only comparative 
studies should be conducted or that other approaches do not provide 
useful information. We ai*e only pointing out that the number of studies 
that evaluate the effectiveness of programs using a comparative analysis 
approach is very limited. We have added language to the report to indicate 
the benefit of other study approaches. 
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18. The Department of Labor expressed concern that gao's analysis did not 
differentiate between the types and sizes of programs. Cur analysis of data 
collection, monitoring, and program effectiveness studies showed that 
larger programs were no more likely to collect outcome data than smaller 
progranfis. Larger programs were more likely to have conducted studies of 
program effectiveness; however, the programs studied only accounted for 
16 percent of the total proposed funding for the 15 programs with 
proposed budgets over $100 million. 

19. The Department of Labor commented that we suggest the desirability 
of linking data on participant outcomes with participant characteristics 
and training provided, but the Department questioned whether such 
linkages can be established for aU programs. The Department states that 
limited resources would preclude such a system in smaller programs. 
While we believe the linkage of participant outcomes with training 
provided and participant characteristics is an important way of 
determining what factors may be influencing program outcomes, we did 
not suggest that every program had the resources to establish such 
information. However, the Department's concern that smaller programs 
may not have the resources to collect data needed to evaluate program 
performance raises a question as to whether we can afford to invest in 
many smaller programs that do not have sufficient resources to evaluate 
their own performance. 

20. The Department of Labor raised several questions concerning the list 
of programs and funding streams that support assistance that enhances 
participant skills or employment opportunities. First, the Department 
questioned why we listed the jtpa IIA and IIC State Education Programs 
separately. The Department states that this is one set-aside program that is 
fimded from two sources. Because we are listing each funding stream 
separately, we listed the jtpa IIA program cenar^tely from the jtpa IIC 
program. Our imderstanding is that state officials must track their funding 
separately for each funding stream, which can create urmecessary 
administrative costs when the two programs are viewed as one program. 
The Department also questioned the funding data for the jtpa Title III 
edwaa program. According to the budget submission dated April 8, 1993, 
the proposed fimding for the Title III edwaa program was $573.7 million. 
The additional dislocated worker ftmding was requested as a part of the 
President's initiative to consolidate employment training for all dislocated 
workers regardless of the cause of dislocation. This proposal has been 
delayed, but additional fimding for dislocated workers was approved. We 
have added a footnote to appendix I to show the significant increase in 
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funding for dislocated workers. The Department also pointed out that the 
JTPA Title ni EDWAA funding goes to substate areas not sdas. We have 
changed the designation in our report to show substate areas. The 
Department also provided updated funding information on several other 
programs. As stated previously, the funding shown in appendix I is 
primarily from the President's proposed budget, dated April 8, 1993, and 
does not reflect the actual funding levels approved by the Congress. 

21. The department suggested a correction for a note in appendix I 
concerning the funding for the Clean Air Act. We have corrected the 
footnote. 

22. The Department questioned why we showed jtpa Title IIA and Title IIC 
in appendix I and did not show them separately in appendix 11. Appendix I 
shows proposed funding levels for fiscal year 1994. The budget for jtpa is 
for program year 1994, starting July 1, 1994, after the separation of the 
disadvantaged adult and youth programs into separate titles. However, 
appendix 11, which shows the extent to which agencies collected outcome 
data, is based on data collected prior to the programs being sepai-ated. The 
same is true for appendixes in and IV. Other studies of jtpa, such as the 
Abt Associates, Inc., which was published after Titles IIA and UC programs 
were separated, still refer to the programs jointly. We have footnoted 
appendixes II, III, and IV to show that the programs were separated after 
July 1, 1993. 

23. The Department of Labor advised us that the jtpa IIA Incentive Grant 
Program does collect outcome data on participant skill attainment. The 
Department also advised us that the Disabled Veterans Outreach Program 
collects participant outcome data on employment status, skill attainment, 
and wage levels. We have corrected appendix n to reflect the outcome 
data collected by these programs. 

24. The Department of Labor reported that through its regional offices, it 
conducts ongoing reviews of jtpa IIA Incentive Program outcomes by state 
and local sda. The Department also stated that the Job Training for the 
Homeless Demonstration Program monitors participant outcomes. We 
have changed appendix III to show this information. 

25. The Department of Labor pointed out that a study shown in appendix 
IV under Job Corps should have been listed under the jobs program. We 
have corrected this error. 
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26. The Department has suggested that we include the Homeless Veterans 
Reintegration Project in our analysis of effectiveness studies shown in 
appendix IV. Because the program was inadvertently omitted from our 
analysis, we do not believe it would be appropriate to include it in one 
section of our work without including it in all sections. Without further 
review of the study, prepared by Technical Assistance Corporation, 
concerning this program, we could not be sure it had been categorized 
correctly. 

27. The Department of Labor expressed concern that our summary of the 
Mathematica study of the Trade A(^ustment Assistance Program was 
taken out of context. We have amended our summary to include more 
information on the Mathematica study. 

28. The Department of Labor questioned our summary of a study of the 
JTPA program in Utah. We have replaced the summary with the correct one. 



Department of 
Transportation 



29. The Depar' lansportation raised a question similar to that of 

several other - ^cncies concerning the funding information in appendix L 
We have added a footnote to appendix I, as well as in the text of the 
report, to show the source of our information was primarily the 
President's proposed budget dated, April 8, 1993. We have also footnoted 
appendix I to show that when information was available, the amounts 
shown have been a(^usted to reflect only that portion of the program that 
served adults and outof-school youth; however, in other instances, the 
funding level shown is for the full progiam, even though only a portion of 
the funding may go to providing employment training as dejfined in this 
report. 
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